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I. 



INTRODUCTION 



A. HISTORY 

Decision making and the process by which decisions are 
made in the complex world of Antisubmarine Warfare (ASW) has 
long confounded the designer as well as the operator of 
modern sophisticated weapons systems. Operator [Zachary 
1980a], software, and hardware [NAVAIR 1975] limitations 
have been cited as causes of less than optimal performance 
in the modern ASW mission. 

Antisubmarine Warfare has traditionally been thought of 
as a half science - half art method of locating hostile 
submerged vessels. Originally, ASW was conducted solely by 
war ships using active ranging equipment where tactical 
'guessing' filled the information gaps left by onboard 
sensors. As the missions of submarines became more 
diversified, air and subsurface weapons platforms were 
developed to augment existing surface weapons systems. With 
the advent of computers, the information gaps of past 
sensors narrowed considerably. However, while the operator 
received more accurate, real time information, the quantity 
of this new information could, at times, overwhelm even a 
seasoned operator. To enhance the operator's ability to 
perform the ASW mission, systems were developed to integrate 
the large quantity of information, filter the 'noise' from 
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the needed inputs, and present the information in an 
orderly, useful manner. Such systems were implemented for 
surface, subsurface, and airborne weapons systems. 

B. ASW MISSION 

All ASW weapons systems - airborne, surface, and 
subsurface - share four basic phases of mission 

accomplishment: search, localization, track, and attack. The 
first phase, search, involves the planning and 

implementation of strategies (sensor placement, areas of 
search, initial tactics) to gain initial detection of 
potential threat. In the second phase, the potential threat 
is localized through the use of additional sensors and 
detection information. After localization of the potential 
threat, the third phase, tracking is accomplished by the 
integration of sensor information and tactics. In peacetime, 
tracking is the mission priority, while in wartime, the 
fourth phase, attacking, receives a' higher priority. These 
phases should be followed systematically to assure, in time 
of peace, constant position information on potential 
threats, or in time of war, destruction of hostile 
submarines . 

Throughout these four phases, ASW personnel are engaged 
in decision making tasks of various degrees of complexity 
and difficulty. Decision aids could be of great use in 
performing these tasks. Antisubmarine Warfare is a dynamic 
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environment in which one can not train for all possible 
conditions. Accordingly, emphasis is placed on software, in 
the form of decision aids, to assist the operator in the 
performance of the ASW mission. However, decision aids have 
traditionally been based on designer subjectivity, random 
fleet inputs, and in response to a new, immediate threat. 
Regretably, fleet ASW personnel have had little impact on 
the design and implementation of ASW decision aids. A 
prioritization of ASW decision areas in which decision aids 
could be of benefit would directly impact on all ASW assets. 

Because of the multitude of ASW platform types in the 
U.S. Navy, the scope of this thesis was limited to one 
platform in the accomplishment of the ASW mission. The 
criterion for selection were (1) an operator whose primary 
duty is ASW, (2) a 'state-of-the-art' weapons system 
(hardware and software) , and (3) ease of data collection. 
The S-3A Viking aircraft fulfilled these criteria and was 
selected as the wearpons platform of interest. 



C. PLATFORM OF INTERESTS 

The S-3A Viking, a carrier based, twin engine jet 
aircraft, with an onboard digital computer, sophisticated 
avionics and crew of four is considered to be the Carrier 
Battle Group's first line of defense from hostile 
submarines. Although the S-3A has other missions, only the 
ASW mission will be addressed in this thesis. The crew of 
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the S-3A consists of: Pilot, who is responsible for safety 
of flight; the Copilot, who operates the nonacoustic sensors 
(Radar, Flir, ESM, and MAD); the Acoustic Sensor Operator 
(SENSO) , who is responsible for passive and active acoustic 
surveillance; and the Tactical Coordinator (TACCO) , who is 
normally the tactical mission commander and is responsible 
for the prosecution of subsurface contacts. Because the 
TACCO receives information from all other members of the 
crew as well as the onboard General Purpose Digital Computer 
(GPDC) for his tactical decision making, he will be of 
primary interest in establishing priority for decision aids. 
For definitive purposes, a decision aid is any methodology, 
algorithm, queue, or filtering mechanism that enhances an 
operator's performance by eliminating unwanted, extranious 
information and presents critical information in an orderly, 
streamline fashion to expedite a needed response. 



D. PURPOSE 



The purpose of this work was to examine ASW decision 
making in the S-3A Viking through the use of mathematical 
modeling techniques and to establish a prioritization 
technique for the development of decision aids to assist 
decision making in the S-3A. 



E. ORGANIZATION OF THIS THESIS 

The main body of this thesis addresses the results of 
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Multidimensional Scaling techniques and the resulting 
applications to decision aids. Chapter 2 discusses the 
advantages and disadvantages of various scaling methods 
classified under the general heading of Multidimensional 
Scaling. The methods evaluated included pairwise comparison, 
unconstrained sorting (Q-Sort), and the triad method. 
Chapter 3 discusses the data collection. Chapter 4 contains 
the data analysis based on the unconstrained sorting 
technique with emphasis on Unfolding Analysis. Finally, 
Chapter 5 discusses the results, conclusions, and 
recommendations . 
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II. MULTIDIMENSIONAL SCALING TECHNIQUES 



A. OVERVIEW 

ASW decision making, because of the complexity and 
quantity of information flow, lends itself to few 
mathematical modeling techniques. ASW decisions, while based 
on a logical flow of events, are subjective in nature 
yielding ordinal data points at best. 

Few modeling techniques can use data of less than 
interval nature. However, a family of modeling techniques 
does exist that use nonmetric (ordinal data) as well as 
metric (interval or ratio data) inputs. One such group of 
techniques is Multidimensional Scaling (MDS) . Before an 
application of Multidimensional Scaling can be used to 
examine ASW decision making, a working understanding of the 
technique should be attained. 

Multidimensional Scaling is the term loosely used to 
identify a large group of varied and powerful techniques for 
the analysis of data normally associated with the behavioral 
sciences. There exist two main purposes for utilizing these 
techniques - (1) to detect any pattern, structure, or 

relationship that may be hidden in a matrix of empirical 
data, and (2) to represent that structure in a form that is 
intuitively appealing and is more intelligible to the human 
(i.e. a two or three dimensional graph vice a five or six 
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degree dimensionalization) . The stimuli under study are 
represented by points in this geometric model such that the 
significant facets of the data points (stimuli) are revealed 
in their relationship among other stimuli [Shepard 1972], 

The spatial representation resembles more traditional 
relationship scales such as temperature or time in that it 
attempts to acquire the fundamental properties (structure) 
of stimuli under study solely by setting them into 
correspondence with positions within a spatial subset of N 
dimensionality. It differs from simple unidimensional (i.e. 
conjoint measurement) scales in that in order to utilize all 
of the information provided by the data or capture the full 
complexity of the stimuli, the data points may assume 
positions within a two, or three dimensional space as well 
[Kruskal 1978] . 

In most cases, one seeks a presentation of the model of 
the lowest possible dimensionality consistent with the 
problem to be solved and the data. Obviously, a lower 
dimensional model would be easier to understand intuitively 
and easier to work with, for it represents the data by means 
of a smaller number of parameters. Minimizing the number of 
parameters also yields a more reliable statistical base due 
to larger data subsets. However, the reduction of 
dimensionality is not an end unto itself. One runs the risk 
of damage to the data by arbitrarily reducing the 
dimensionality of the model. If the relationship among 
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stimuli is sufficiently complex, two or three dimensions, 
then the model better 'fits' the empirical data. The 
goodness of fit measurement called 'stress' is the parameter 
that measures the deviation of the empirical data from the 
model. There exist a gray area of trade-offs between easy 
visualization and the use of all data information. It is the 
analyst's responsibility to execute the trade-off after 
evaluating the criteria for analyzing the model. 

Multidimensional Scaling utilizes the same basic 
concepts as Factor Analysis. The high dimensionality, 
usually associated with Factor Analysis, is caused by the 
rigid assumption of linearity among stimuli. This assumption 
is relaxed in the model used for Multideminsional Scaling, 
allowing (normally) a two or three dimensional 
representation. Essentially, Multidimensional Scaling 
provides greater stimuli resolution in a more readily 
visualizable model than Factor Analysis. 

Multidimensional Scaling uses similiari ties or 
dissimil iari ties among stimuli to define the relationships 
that exist in each possible dimension. Several subsets of 
multidimensional scales exist in various forms for different 
appilcations . The subsets include paired and triad 
comparisons, unconstrained sorting, conjoint measurement, 
delphi method, and multiple attribute utility models. 
However, the major subsets considered in this thesis were 
limited to pairwise comparison, unconstrained sorting, and 
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triad comparison because of external constraints placed on 
the data collection. The constraints were the short periods 
of time available for data collection and the 
inaccessabili ty of subjects after the initial interview. The 
constraints will be discussed throughly in Chapter III. 
Because of the multiple passes required through the data and 
subject interaction required, the techniques of multiple 
attribute utility, delphi and conjoint measurement were 
considered less desirable than the selected techniques. 

The measure of effectiveness for the pairwise 
comparison, unconstrained sort, and triad comparison is how 
well the model or technique represents the empirical data. 
This measure of effectiveness is called 'stress' [Kruskal 
1978] . Stress is the goodness of fit measurement for the 
model used to determine the number of dimensions considered, 
balanced by the amount of deviation from the data. Stress 
measures the degree of departure from the assumption that 
there is a monotonic relationship between the non-metric 
dissimiliarity measures of the data and the metric distance 
measures of the representational structure [Burton 1968]. 

B. PAIRWISE COMPARISON 

The most common technique is the pairwise comparison of 
stimuli with a ranking or weighting between the two [Kruskal 
1978]. The pairwise comparison examines N different stimuli 
in pairs, assigning relative ranks as to the degree of 
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similiarity or dissimiliari ty of the two stimuli being 
compared. The comparisons are used to form matrices of 
ordinal preference data. The pair-wise data is examined for 
hidden relationships among three or more stimuli that would 
not normally come to light in pair-wise preference mapping. 
These relationships form the dimensionality of the model and 
point to areas of greatest (or least) interest. 

While the pairwise technique utilizes the greatest 
amount of information from a given set of data, several 
limitations become apparent in the use of this comparison 
method. For N stimuli, the number of pair-wise comparisons 
are N (N - l)/2. For all but a small number of comparisons 
(N _< 10) , the number becomes unmanageable. The complexity 

increases with larger subject samples. Additionally, the 
tester normally has a finite time limit and even a small 
sample (N = 8, subjects = 10) requires a large amount of 
time . 

C. UNCONSTRAINED SORTING 

The unconstrained sorting task [Burton 1975] requires 
subjects to sort stimuli into groups. The sorting into 
groups represents a partitioning of the stimuli set. 
Normally, the stimuli set consist of the stimuli represented 
in a medium that is easily sorted (such as 3 X 5 cards) and 
test subjects are asked to sort the stimuli into groups. The 
instructions for the formation of these groups require the 
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subjects to sort the stimuli that are 'similar in meaning' 
or 'belong together' into the same group. The subjective 
interpretation of the sorting instructions by each member is 
the basis for an unconstrained sort. The subsets (groups) of 
stimuli can have any number of members. A single stimulus in 
a group represents no similarity to any other stimulus and 
all stimuli in a group represent total similarity among 
stimuli. Test subjects who have few subgroups of stimuli are 
called 'lumpers' and subjects who make many distinctions 

among stimuli are referred to as 'slitters'. Burton's study 
of Q-Sort data [1975] suggest that among Multidimensional 
Scaling techniques, the unconstrained sorting task is the 
most flexible in terms of testing procedures. 

D. TRIAD TEST 

The triad test is similar in procedure to the paired 
comparisons discussed earlier except stimuli are compared 
three vice two at a time. The triads test may be 

administered in two different ways. The first way requires 
the subject to choose the most different stimulus from among 
the three stimuli presented. The second version of the test 
asks the subject to pick the most different stimulus (as 
before) and also to pick the pair of stimuli which are most 
different (one from another) [Burton 1975]. 

There are ( 3 ) or N (N - 1) (N - 2)/6 triads for N 

stimuli. These triads are used to form matrices of ordinal 
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preference data. This ordinal data is monotonically 
transformed to a metric form that can be examined for 
relationship or structures that are not apparent at first 
inspection . 

E. COMPARISON OF TECHNIQUES 

Although the pairwise comparison was the most common 
test found in the literature review [Shepard 1972] , [Kruskal 
1978] , the unconstrained sort and triads test revealed 
several interesting advantages for the implementation of 
either technique. The advantages and disadvantages of each 
technique are discussed in this section. 

All three techniques share the advantages of being a 
simple task to understand and all can be presented either 
verbally or in written form. At this point, the paired 
comparision and triads test begin to differ from the 
constrained sort. For N stimuli and M subjects, the number 
of pairs presented by the tester is ( ^ ) M or 

N (N - l)M/2 and the number of triads presented equal ( 3 ) M 
or N (N - 1) (N - 2) M/ 6 . Intuitively, one can see the 

advantage of the triads test in the sheer number of stimuli 
groups. However, for N 10 and M 10, either test soon 
becomes unmanageable in terms of time necessary to give the 
test . 

On the other hand, the unconstrained sorting technique 
has one clear advantage : time. The unconstrained sort can 
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be given to several subjects at once and recent research by 
Burton [1975] suggests that most people can do a single 
sorting of sixty(60) stimuli in fifteen to thirty minutes. 
Additionally, experiments by Miller [1974] indicate that 
triads test and sorting task uncover the same structure 
within sorting decisions. 



F. SELECTION OF TECHNIQUE 

As discussed -in the preceeding sections, each of the 
three Multidimensional Scaling techniques can be applied to 
a great number of problem areas with varying degrees of 
success. An examination of the problem area suggested that 
one technique was superior to the other two. 

The problem concerned the ASW operational decision 
making environment, specifically the S-3A aircraft. Because 
of the dynamics of the S-3A environment ashore (Weapons 
Systems Trainers, ground schools, and training flights), it 
was essential to interview as many experienced TACCOs as 
possible in the shortest amount of time so as not to 
interfere with training and operational commitments. 
Consequently, with time as the major constraint, the 
unconstrained sorting technique was selected. 
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III. APPLICATION 



A. PROBLEM 

The development of air ASW decision aids requires an 
understanding of the complexities and the interrelationships 
among decisions of the ASW mission. The measure of 

importance with respect to ASW decision making is a 

non-quant i f i abl e variable that directly impacts on 

priorities among decisions. Because of the subjective nature 
of ASW decision making. Multidimensional Scaling, in 

general, and the Unconstrained Sorting Task, specifically, 
lend themselves favorably to the prioritization of the ASW 
decision space. An examination of the specific S-3A ASW 
decision making process was helpful as an overview to the 
problem. 

As mentioned in the introduction, the air ASW mission 
was divided into four general phases: (1) search, (2) 

localization, (3) track, and (4) attack. While all of these 
areas are important, the number of stimuli (phases) 
concerning the mission was too small to yield any 
appreciable information about the decision function. An 
improvement over the four phase model was the repartitioning 
by Zachary [1980a] of the air ASW decision space into six 
specific decision situations. The six specific decision 
situations were: 
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1) On-station search 

2) Contact classification/verification 

3) Localization 

4) Surveillance tracking 

5) Attack planning 

6) Lost contact requisition 

The six decision situations model, while an improvement 
over the four phase model, did not remedy the problem of 
limited information used as input to Multidimensional 
Scaling models. The six 'phase' model was further evaluated 
by Zachary [1980b] and found to have varying numbers of 
constituent decision functions associated with each phase. 
These constituent decision functions were not considered to 
be confined to only one decision situation but rather each 
decision function could be used in any or all combinations 
of the decision situation (phases) to which that decision 
function was considered applicable. Table I shows the 
decision function composition of the six air ASW decision 
situations . 

From a careful examination of the decision situations in 
Table I, fourteen (14) distinct decision functions emerged 
as necessary to completely describe the air ASW decision 
space within the context of this thesis. Consequently, the 
fourteen decision functions listed in Table II became the 
stimulus set for the multidimensional scaling sampling. The 
decision functions and their descriptions are listed in 
Table II. 
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Table I 



Decision Function Composition of Six 
Air ASW Decision Functions 



DECISION SITUATION 


CONSTITUENT DECISION FUNCTIONS 


ON-STATION SEARCH 


CREATE SENSOR PATTERN 

MANAGE EQUIPMENT/STORES 

EXTEND SENSOR PATTERN 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 

COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 


CONTACT CLASSIFICATION/VERIFICATION 


CLASSIFY SIGNAL 

DETERMINE SIGNAL IS A VALID CONTACT 


LOCALIZATION 


MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

GAIN ATTACK CRITERIA 

DETERMINE TARGET FIX 

COORDINATE HAND-OFF 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 

COMPENSATE FOR JN-SITU PROPAGATION CONDITIONS 


SURVEILLANCE TRACKING 


CREATE SENSOR PATTERN 

MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

DETERMINE TARGET FIX 

COORDINATE HAND-OFF 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 

COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 


ATTACK PLANNING 


CREATE SENSOR PATTERN 

MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

GAIN ATTACK CRITERIA 

DETERMINE TARGET FIX 

ADJUST PATTERN TO SENSOR FAILURE 

DETERMINE WEAPON AND SETTING FOR ATTACK 

CONSTRUCT SENSOR MONITORING PATTERN 

DETERMINE AIRCRAFT WEAPON LAUNCH POSITION 

COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 


LOST CONTACT REACQUISITION 


CREATE SENSOR PATTERN 

MANAGE EQUIPMENT/STORES 

ANTICIPATE TARGET MOVEMENT 

EXTEND SENSOR PATTERN 

COORDINATE HAND-OFF 

ADJUST PATTERN TO SENSOR FAILURE 

CONSTRUCT SENSOR MONITORING PATTERN 

COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 
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Table II. Description of the Fourteen Decision Functions 

- ADJUST PATTERN TO SENSOR FAILURE (AP) 
replacement of faulty sensors or adjustment of 
aircraft track due to requipment/sensor failure. 

- EXTEND EXISTING SENSOR PATTERN (EP) - determine 
orientation and settings of sensors to be added to 
existing pattern with no contact. 

- ANTICIPATE TARGET MOVEMENT (AM) - predict future 
location, course, speed, and depth of target based 
on intelligence and actual tactical situation. 

- CONSTRUCT SENSOR MONITORING PATTERN (MP) - 
determine where to position the aircraft to obtain 
maximum reception from sensors, and sequence in 
which to monitor deployed (acoustic) sensors if 
number of sensors exceeds number of aircraft 
receivers . 

- COORDINATE HAND-OFF (CH) - transmission of data 
to relief platform for continuation of prosecution. 

- DETERMINE SIGNAL IS A VALID CONTACT (VC) - reduce 
false alarm rates of sensor system (e.g., clouds on 
radar, random noise on MAD or acoustic). 

- DETERMINE WEAPON AND SETTING FOR ATTACK (DS) - 
select weapon to be used for the attack, and its 
optimum weapon settings (e.g., search depth, 
minimum/maximum search limit) . 

- GAIN ATTACK CRITERIA (AC) - interpret sensor data 
to determine when target is localized sufficently 
to place an attack. 

- DETERMINE TARGET FIX (TF) - use incoming sensor 
data to establish the location of the target. 

- CREATE SENSOR PATTERN (CP) - determine pattern, 
spacing, orientation and/or utilization for all 
sensor types for each phase of the mission. 

- MANAGE EQUIPMENT/STORES TO ACCOMODATE PRESENT AND 
FUTURE NEEDS (ME) - monitor inventory of sensors 
and equipment status for application for future 
tactics and resources. 
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Table II. Continued 



- CLASSIFY SIGNAL (CS) - determine if signal 
(primarily acoustic) may be originating from the 
target of interest. 

- COMPENSATE FOR ACOUSTIC/ATMOSPHERIC PROPAGATION 
CONDITIONS (PC) - determine the adjustments in 
sensor spacing, aircraft track, aircraft altitude, 
etc., that must be made when actual (in situation) 
atmospheric and bathythermal conditions are 
different from their forecast. 

- DETERMINE AIRCRAFT WEAPON LAUNCH POSITION (LP) - 
determine factors to place the aircraft in an 
optimum attack position. 

In this analysis, the TACCOs were asked to perform two 
psychometric tasks to elicit indicators as to the subjective 
nature of ASW decision making. The first task was an 
unconstrained sorting of the stimulus set of 14 decision 
functions that provided the necessary inputs for the 
Multidimensional Scaling. The second task was a set of rank 
orderings of the stimulus set that provided inputs to the 
unfolding analysis and the correlations testing. Both task 
will be discussed in detail in the following sections. 

1. Unconstrained Sorting Task 

The technique of the unconstrained sorting, also 
know as the 'Q-sort' , was first developed by Stephenson 
[1953]. The technique was further refined by Burton [1968] 
and applications of the scaling were validated by Miller 
[1974]. In this task, the subject is asked to partition the 
members of the stimulus set into an arbitrary number of 
groups of unspecified size. The criterion for sorting 
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usually is a similarity or dissimiliar i ty judgement by the 
subject established by the instructions for the task. There 
is no limit on the number of stimuli in a group nor is there 
a limit on the number of groups. 

As discussed in the previous chapter, the 
unconstrained sorting task is only one of many psychometric 
methods used to collect data for Multidimensional Scaling. 
Unconstrained sorting was chosen over the other discussed 
techniques for two reasons. First, the unconstrained sort 
allows the levels of discrimination among the stimuli by 
each subject to be controlled for explicitly in the 
processing of the results. Subjects will place varying 
degrees of emphasis on the underlying structure of the 
interrelationships among stimuli. The algorithms developed 
for this technique in the processing of data inputs control 
for between subject variance. Subjects, however, vary widely 
in the level of detail perceived to be necessary for 
performance of the sorting task. The perceived level of 
detail task performance can differentiate broad distinctions 
that find all stimuli very much alike or very different; or 
from very few distinctions that find small groups of stimuli 
similiar but in different degrees. Depending on the degree 
of distinction required by the interviewer, the instructions 
must be expressed so as to focus on a specific level of 
distinction required of a scaling task. This is the primary 
reason for choosing the unconstrained sorting task. 
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Secondly, the unconstrained sort is the simplest, 
most expeditious method of collecting data for 
Multidimensional Scaling. As discussed in the previous 
chapter, the limited amount of time available for TACCO 
interviews was critical. The sorting task portion of the 
interview took less than 30 minutes for each TACCO, favoring 

this technique over more time consuming methods. 

Thorough investigation of Burton [1968], Miller 
[1974] and Zachary [1980b] have shown that unconstrained 
sorting produces equivalent results to the other MDS 
techniques when two general rules of thumb are obeyed. The 
first rule is that the number of stimuli must be greater 

than ten, (S > 10), and second, there must be at least twice 

as many subjects (N) performing the sort as there are 
stimuli, (N 2S) . The fourteen decision functions, 
previously discussed, meet the stimuli number requirement 
and a total of 30 S-3A TACCOs interviewed satisfied the 
second condition. 

Each TACCO interviewed was presented the sorting 
task stimuli verbally, in the briefing, and in writing, in 
the form of a deck of 3X5 inch index cards. The written and 
the verbal presentations were identical in nature. In the 
center of each stimulus card (in bold print) was the name of 
the decision. Following the name was a brief explanation of 
the decision function. To identify each stimulus card, a two 
letter code was used (in the upper right corner) to record 
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the sorting response as well as the ranking (to be discussed 
later). Figure 1 represents a sample decision card used in 



the interviews. The 
from the keywords of 
the codes and the 
collection. Mnemonic 
or alphabetic codes 
cards were sorted 
ordering . 



codes were mnemonic in nature, derived 
the decrision functions. Table II lists 
decision functions used in the data 
codes were selected instead of numeric 
to eliminate any bias in the way the 
due to any unintentional implicit 



Figure 1. Sample Decision Function Card 



AM 



ANTICIPATE TARGET MOVEMENT 



predict future location, 
depth of target based on 
actual tactical 



course, speed, and 
intelligence and 
situation 



2 . 

was a 
cards , 



Ranking Task 

The second psychometric task performed by the TACCOs 
rank ordering task. The subjects used the stimulus 
discussed previously, and ranked the stimuli 
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according to four different criteria. The criteria for the 
four rankings were: 

1) Importance of the decision in the ASW mission 
with an attack on a submarine. 

2) Importance of the decision in the ASW mission 
with only surveillance of a submarine. 

3) Urgency of the decision, and 

4) TACCO workload during the decision. 

Rankings 1 and 2 provided inputs for the Unfolding 
Analysis (discussed in Chapter IV). They were selected for 
the Unfolding Analysis because they use the same criterion - 
importance of the decision to the accomplishment of mission 
but under different circumstances - attack versus 
surveillance. This difference was used as a reference. The 
criterion of importance was defined as the degree to which a 
less than optimal decision would adversely impact on the 
achievement of the mission objective. 

Urgency, the third ranking criterion, was defined as 
the relative speed with which the decision had to be made, 
at a decision point. The fourth ranking criterion, workload, 
was defined only in the most general terms. Each TACCO was 
instructed to consider mental as well as physical workload. 
These last two rankings were used to provide supplemental 
information which could aid in the interpretation of the MDS 
solution . 
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B. DATA COLLECTION 

This thesis is based on the assumption that a 
priorization of the S-3A ASW decision space must include the 
knowledge, opinion, and intuition of experienced S-3A ASW 
decision makers (TACCOs) . Multidimensional Scaling and 
Unfolding Analysis, discussed in the data analysis section 
of this thesis, provided a way for numerically prioritizing 
the fourteen decision functions previously discussed, on the 
basis of structured but subjective inputs from S-3A TACCOs. 

The primary source of experienced S-3A TACCO's for this 
analysis was VSWING 1, NAS Cecil Field, Jacksonville, 
Florida. For data collection purposes, an experienced TACCO 
was defined as an individual who had been designated an S-3A 
TACCO for at least one year or had made at least one 
deployment as a qualified S-3A TACCO. VSWING 1 was selected 
as the sample TACCO population primarily because of the 
availability of experienced TACCOs during the limited time 
frame of the data collection effort. 

The structure of the Multidimensional Scaling interview 
consisted of three general parts: 1) initial briefing and 

explanation, 2)sorting task, and 3) ranking task. At the 
conclusion of the interview an open-ended discussion of 
decision aids and decision making was held at the 
interviewee's option. The interview was designed to allow 
several TACCO's to be interviewed simultaneously but 
conversation between TACCOs was prevented. The interviews 
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were designed to be completed in a two hour period. 



as 



to 



sample bias due to interviewee fatigue and to 
work time loss. The designed schedule was as 

- 30 minutes were allowed for initial briefing and 
answering questions 

- 20 minutes were allocated for the psychometric 
sorting task 

- 40 minutes were allocated for the psychometric 
ranking task 

- 30 minutes were used to discuss the interview and 
suggested improvements in the area of decision 
aids . 

It should be noted that the TACCO's interviewed demonstrated 
an active interest in providing fleet inputs to an area many 
believed to be critical to the successful accomplishment of 
the S-3A ASW mission. 

At the start of each interview, an initial briefing and 
explanation was given, outlining air ASW history, purpose of 
the interview and the possible impact of decision 
prioritization on the implementation of the future decision 
aids in the S-3A. Questions were then answered and the 
interviewees were introduced to the two tasks they were to 
perform. Portions of the briefing are found in Appendix A. 

The actual data collection effort for the sorting and 
ranking tasks were performed as follows. After the briefing. 



minimize 
minimize 
follows : 
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each TACCO was given a pack of cards (similiar to Figure 1) 
and a set of response forms (reproduced in Appendix B) . The 
cover sheet of the response form requested background 
information on each TACCO but the name was purposely left 
off. The absence of names from the response forms preserved 
anonimity, hopefully leading to less biased responses. The 
background information allowed the interview to insure the 
requirement of experience and training were met prior to 
inclusion in the test sample. Each task had an instruction 
sheet preceeding each response form, explaining the required 
task once again. The unconstrained sort used perceived 
simil iari ties among stimuli as the criterion for group 
partitioning. The four ranking task used the criterion 
discussed previously. Individual questions during the task 
were discouraged to decrease any answer interpretation bias. 
All subjects finished the sorting task in less than 15 
minutes, while the rankings, normally, required 10 minutes 
per task. 

Following the last ranking task, each TACCO had an 
opportunity to comment, regarding ASW decision aids in 
general, and S-3A decision aids in particular. Comments were 
solicited concerning the interview (new ideas, format, 
procedure) and are discussed in the Result and Conclusion 
sections of this thesis. 
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IV. DATA ANALYSIS 



A. OVERVIEW 

Chapter Four describes the data analysis for this 
effort. Initially, the sorting data was preprocessed using 
Burton's [1975] algorithm into a dissimiliarii ty matrix to 
be used as input into the MDS program. The ranking data was 
analyzed through the use of correlation techniques and 
average rankings were found. The dissimiliar ity matrix was 
then input into the MDS program and a dimensional 
representation of the decision space was produced. 
Dimensional interpretations were derived. Next, Unfolding 
Analysis was used to provide a representative model of the 
ranking decision space. Finally, a priority scale of the 
decision space was established using the model coefficients 
of the Unfolding Analysis and the Multidimensional Scaling 
coordinates. Figure 2 provides an overview of this 
methodology. 

B. PRELIMINARY ANALYSIS 

1 . Preprocessing of Sorting Data 

Sorting data were obtained from 32 experienced S-3A 
TACCOs, however, two TACCOs were excluded because they did 
not have the experience level required for the analysis. The 
remaining 30 TACCOs were determined to be 'experienced' for 
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Figure 2. Multidimensional Scaling Prioritization 
Procedure (adapted from Zachary 1980b) 



the purposes of this analysis. These 30 sortings are listed 
in Appendix C. The 30 sets of data were prepared for input 
into the MDS computer program by METRIC, an algorithm 
developed by Burton (1975) . The algorithm processes the 
unconstrained sorting data to produce a dissimil iari ty 
measure among the decision functions in the stimulus set 
[Zachary 1980b]. In general, a dissimiliarity measure is a 
binary mapping from the set of objects scaled to some 
numerical scale. For any two objects i and j, the 
dissimiliarity between i and j is expressed as d^j. 
Dissimiliarity measures has three main properties: 

1) d— = 0 iff: i = j : positivity 
d.^ > 0 otherwise 

2) d^j = d^j for all i, j : symmetry 

3) d ij i. d ij + for all i , j , k : triangle 

inequality. 

Dissimiliarity measures are normally represented as 
a square matrix where each row/column represents one of the 
stimuli. Consequently, the terms dissimiliarity 'measure' 
and dissimiliarity 'matrix' are use throughout the analysis 
interchangeably. 

Burton's algorithm computes dissimiliarity between 
two stimuli based on two inputs: 

1) the number of subjects who place them in 
different groups, and 
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2) the values assigned to the parameters a 
and ( 

The first parameter, a , represents the level of 
discrimination selected for this analysis (as discussed in 
Chapter III). The parameter, ( , represents the minimal 

dissimiliari ty between two stimuli that results when two 
stimuli are placed in the same group by one subject. The 
algorithm is based on the principle that each subject infers 
a partition on the set of stimuli. R. is defined as the 
number of groups formed by subject i and is defined as 
the number of stimuli placed in group k by subject i. If the 
number of subjects is T, then dissimiliari ty between stimuli 
x and y is: 

d xy = Z (c ‘ s i (x ’ i' )) 



where : 



Sj (x, y) 



if subject i places stimuli x and 
y in cell k of the partition, 

B- if subject i places stimuli x and y 
in different cells of the partition, 
and 

C if x = y. 



d(x, y) is a measure of dissimiliarity which, in addition, 
is a metric [Burton 1975]. There are two constraints on d: 

1) B^ < min (A.^) over all k 



39 



2) C > max (A^) over all i , k 
Additionally, a useful measure is obtained by defining: 

< M ik> 

0 

1 + ( for a = 0 

N + < for a > 0 

2 + ( for a < o 

Since epsilon ( ( ) is a minimum value used to 
differentiate items placed together by a given subject, it 
should be very small. Alpha ( a ) represents the level of 
distinction used by the TACCO doing the sorting. If alpha is 
greater than zero ( a > 0) , a greater weight is given to 
sortings that use a few, large cells, and the distance 
measure tends to be the lower dimensionality when subjected 
to Multidimensional Scaling. When alpha is less than zero 
( a < 0) , a greater weight is given to partitions which 

include a greater number of small cells, and the measure 
results in higher dimensional MDS solutions. When alpha 
equals zero ( a = 0) , equal weight is given large and small 
cell partitions, as 1 is always added to d(x,y) whenever x 
and y are in different cells, and epsilon is added to d(x,y) 
when they are place in the same cell. 

The program, METRIC, performs this algorithm to 
produce a d i ss im i 1 i a r i ty matrix from the unconstrained 
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sorting. Because the intent of this thesis is to identify 
the most important dimensions vice all dimensions, an alpha 
value of -.5 was used. The dissimiliari ty matrix is shown 
in Table III. 

2. Preliminary Analysis of Rankings 

The rankings of the 14 decision functions by the 
four criteria discussed in Chapter 3 are given in Appendix D 
for all 30 TACCOs . The requirement of the preprocessing of 
data, necessary in the sorting data, was not considered 
necessary for input into the Unfolding Analysis procedure. 
However, two preliminary analyses of the ranking data were 
considered useful to the overall analysis. 

The first preliminary analysis tested the 
statistical significance of the ranking data. The Kendall 
coefficient of concordance : W is a statistic which measures 
the relation (degree of agreement) among several rankings 
of the same set of stimuli [Seigal 1956]. The coefficient of 
concordance is an index of the divergence of the actual 
agreement shown in the data from the maximum possible 
(perfect) agreement. A test of significance of the 
coefficient can be based on the value of W [Kendall, 1962]. 
This test is based on the fact that K (N - 1)W is distributed 
according to the Chi-Square distribution with N - 1 degrees 
of freedom (where K is the number of rankers and N is the 
number of stimuli). The null hypothesis, H q , and the 
alternate hypothesis used were: 
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Hq : there is no agreement among the subject's 

rankings (i.e. W = 0) . 

: there is significant agreement among the 

subject's rankings (i.e. W > 0) . 

Table IV lists the coefficient of concordance : W, 

2 

the X statistic, and the level of significance for each of 
the four rankings. All four rankings were found to be 
significant at less than the .01 level, rejecting the null 
hypothesis with greater than 99 percent confidence. The 
significance testing indicates that: 

1) the TACCOs used the same criterion for ranking, 
and 

2) the response were not of a random nature. 

The second preliminary analysis of the rankings 
provided insight into the interpretation of each criterion 
(i.e. 1) importance to mission with attack, 2) importance to 

mission without attack, 3)urgency, and 4)workload). Average 
rankings were used as a tool for this analysis. Siegal 
[1956] and Kendall [1962] suggest that the best estimate of 
the 'true' ranking of N objects is provided, when W is 
significant, by the order of the various sums of the ranks. 
An extention of this estimate is the average ranking. 
Average rankings yield more information about relative 
ranking among stimuli than does the sum of ranks [Zachary 
1980b] . The average rankings are shown in Tables V through 
VIII. The tables show a significantly different ranking for 
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each criterion, verifying the assumption that more than one 



criterion was necessary to capture the implicit 
prioritization of the decision functions by the TACCOs. 



Table IV. 


Significance 


of Subject 


Agreement in Ranking 


RANKING 

CRITERION 


KENDALL 

COEFFICIENT 


CHI-SQUARE 

STATISTIC 


SIGNIFICANCE LEVEL 
(FOR REJECTION OF H Q ) 


IMPORTANCE 

WITH 

ATTACK 


.382 


148.79 


< .001 


IMPORTANCE 

WITHOUT 

ATTACK 


.468 


182.56 


< .001 


URGENCY 


.558 


217.62 


< .001 


WORKLOAD 


.243 


94.80 


< .001 
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Table V. Average Ranking of Decision in Mission 

With Attack 



Gain Attack Criteria 3.866 

Classify Signal 4.700 

Determine Tarrget Fix 5.400 

Determine Aircraft Weapon Launch 
Position 5.433 

Determine Weapons Setting For 

Attack 5.800 

Anticipate Target Movement 6.200 

Determine Signal Is Valid Contact 6.566 

Create Sensor Pattern 6.600 

Compensate For Propagation Conditions 8.500 
Construct Sensor Monitoring Pattern 8.666 

Extend Sensor Pattern 10.033 

Manage Equipment and Stores 10.200 

Adjust Pattern to Sensor Failure 10.566 

Coordinate Hand-Off 12.533 



Table VI. Average Ranking of Decision in Mission 

Without Attack 



Classify Signal 3.900 

Anticipate Target Movement 4.966 

Create Sensor Pattern 5.066 

Determine Signal Is Valid Contact 5.166 

Determine Target Fix 5.266 

Coordinate Hand-Off 5.666 

Compensate For Propagation Conditions 6.500 
Construct Sensor Monitoring Pattern 7.233 

Manage Equipment and Stores 7.766 

Extend Sensor Pattern 8.366 

Adjust Pattern to Sensor Failure 8.733 

Gain Atack Criteria 10.600 

Determine Aircraft Weapon Launch 
Position 13.033 

Determine Weapon and Setting 

For Attack 13.066 
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Table VII. Average Ranking of Decision by Urgency 



Determine Aircraft Weapon Launch 
Position 2.666 

Gain Attack Criteria 2.933 

Determine Weapon and Setting 

For Attack 4.800 

Anticipate Target Movement 5.266 

Determine Target Fix 5.333 

Determine Signal Is Valid Contact 5.766 

Classify Signal 6.866 

Adjust Pattern to Sensor Failure 8.400 

Manage Equipment and Stores 9.366 

Compensate For Propagation Conditions 9.900 
Extend Sensor Pattern 10.266 

Create Sensor Pattern 10.666 

Coordinate Hand-Off 11.366 

Construct Sensor Monitoring Pattern 11.933 



Table VIII. Average Ranking of Decision by Workload 



Gain Attack Criteria 4.266 

Determine Target Fix 4.733 

Anticipate Target Movement 5.333 

Determine Aircraft Weapon Launch 
Position 5.566 

Determine Weapon and Setting 

For Attack 6.500 

Coordinate Hand-Off 7.000 

Create Sensor Pattern 7.300 

Manage Equipment and Stores 7.600 

Extend Sensor Pattern 8.166 

Adjust Pattern to Sensor Failure 8.633 

Compensate For Propagation Conditions 9.100 
Construct Sensor Monitoring Pattern 10.166 

Determine Signal Is Valid Contact 10.333 

Classify Signal 10.333 
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C. APPLICATION OF MDS ALGORITHM 

The dissimiliari ty matrix discussed in the preliminary 
analysis was the primary input to a MDS program. The 
analysis was conducted on the University of Pennsylvania 
Wharton School's DEC-10 computer using the MDSX package of 
Multidimensional Scaling programs [Coxon et al 1977], This 
package contains several different algorithms including the 
MINASSA algorithm [Lingoes and Roskam 1975] , the INDSCAL 
algorithm [Carrol and Chang 1970] and the TORSCA IV 
algorithm [Young 1968], Zachary [1980b] used each of these 
algorithms on similar ASW data to test for convergent 
validity among the programs, resulting in virtually 
identical solutions. Therefore, in the dimensional and 
prioritization analysis, the MINASSA algorithm was used 
because the other two algorithms are quasi-metric, whereas 
MINASSA is totally nonmetric, that is, useful in analysing 
ordinal data. 

The MINASSA algorithm was used to construct solutions 
from one to five dimensions. The plot of the goodness of fit 
measure 'stress' versus dimension number, shown in 
Figure 3, indicates that a three dimensional solution is 
best. Kruskal [1978] suggest that if the number of stimuli 
considered, N, is greater than four times the dimension 
number, D, then a stress measurement of .02 or less is 
considered a very good fit. Additionally, Roskam's [1977] 
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stress 




number of dimensions 

Figure 3. MDS Stress vs. Dimensionality 
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'rule-of-thumb' for assessing the stress of a MINASSA 
solution indicates that a stress between .05 and .01 is 
considered very good quality. Therefore, with a stress of 
.018, the three dimensional solution satifies the stress 
requirement for a good fitting model. Although the 4th and 
5th dimensional solutions also provide stress measurements 
less than .02, the incresed complexity of the solutions due 
to higher dimensionality did not merit further 
consideration . 

The three dimensional solution is shown in Figure 4. For 
ease of interpretation, two dimensional plots of each 
possible pair of dimensions are given in Figures 5 through 
7. The plots represent relative locations in two dimensions 
of each of the 14 decision functions. Table IX shows the 
actual coordinates of each decision function in three 
dimensions . 



D. DIMENSIONAL INTERPRETATION 

The MINASSA algorithm reconstructs the underlying 
features of relationships of the decision functions as the 
dimensional axes of the MDS solution. Unfortunately, the 
algorithm cannot identify the meaning of these dimensional 
axes. The meanings of the axes must be interpreted through 
careful examination of the axes individually, through 
unidimensional projection of each coordinate set, and the 
interaction of the axes. The unidimensional projections are 
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shown in Figures 8 through 10. 

The multidimensional representation constructed by 
MINASSA is unique and translation or rotation of the axes 
used to obtain a more interpretable set of projections did 
not change the relationship among sitmuli. However, the 
interpretations of the dimensions, discuss subsequently, are 
based on the 'unrotated - untranslated' spatial 
representations given in Figures 5 through 7. The basis for 
this representation is two-fold. First, the MINASSA solution 
is based on an implicit rotation performed by the algorithm 
to minimize dependence among dimensions. The orientation of 
the dimension axes provided the most independent 
representation in terms of the set of decision functions. 
Consequently, the independent nature of the resulting axes 
enhanced the interpretabili ty of the dimensions. Secondly, a 
rotation and translation of the axis was attempted to 
provide a more interpretable representation. This analysis 
did not add to the interpretabil i ty of the spatial 
representation. Additionally, because the unrotated 
orientations is based on the criterion of minimizing the 
interdimensional correlation, the original MINASSA result is 
used in the following interpretation. 
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Figure 4. Three Dimensional MDS Plot 



DIMENSION 



- © 
© 

© 


% 


© ® 


© 

© 1 © 

© 



DIMENSION 2 



KEY 

T. ADJUST PATTERN TO SENSOR FAILURE 

2. EXTEND SENSOR PATTERN 

3. ANTICIPATE TARGET MOVEMENT 

4. CONSTRUCT SENSOR MONITORING PATTERN 

5. COORDINATE HAND-OFF 

6. DETERMINE SIGNAL IS VALID CONTACT 

7. DETERMINE WEAPON AND SETTING FOR ATTACK 

8. GAIN ATTACK CRITERIA 

9. DETERMINE TARGET FIX 

10. CREATE SENSOR PATTERN 

11. MANAGE EQUIPMENT/STORES 

12. CLASSIFY SIGNAL 

13. COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 

14. DETERMINE AIRCRAFT WEAPON LAUNCH POISITCN 



Figure 5. Dimension 1 vs Dimension 2 
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DIMENSION 




KEY 

1. ADJUST PATTERN TO SENSOR FAILURE 

2. EXTEND SENSOR PATTERN 

3. ANTICIPATE TARGET MOVEMENT 

4. CONSTRUCT SENSOR MONITORING PATTERN 

5. COORDINATE HAND-CFF 

6. DETERMINE SIGNAL IS VALID CONTACT 

7. DETERMINE WEAPON AND SETTING FOR ATTACK 

8. GAIN ATTACK CRITERIA 

9. DETERMINE TARGET FIX 

10. CREATE SENSOR PATTERN 

11. MANAGE EQUIPMENT/ STORES 

12. CLASSIFY SIGNAL 

13. COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 

14. DETERMINE AIRCRAFT WEAPON LAUNCH POISITCN 



Figure 6. Dimension 1 vs Dimension 3 
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DIMENSION 3 



KEY 

T. ADJUST PATTERN TO SENSOR FAILURE 

2. EXTEND SENSOR PATTERN 

3. ANTICIPATE TARGET MOVEMENT 

4. CONSTRUCT SENSOR MONITORING PATTERN 

5. COORDINATE HAND-OFF 

6. DETERMINE SIGNAL IS VALID CONTACT 

7. DETERMINE WEAPON AND SETTING FOR ATTACK 

8. GAIN ATTACK CRITERIA 

9. DETERMINE TARGET FIX 

10. CREATE SENSOR PATTERN 

11. MANAGE EQUIPMENT/STORES 

12. CLASSIFY SIGNAL 

13. COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 

14. DETERMINE AIRCRAFT WEAPON LAUNCH POISITON 



Figure 7. 



Dimension 2 vs Dimension 3 
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Table IX. Final Multidimensional Scaling Coordinates 





Dimension 


Dimension 


Dimension 




1 


2 


3 


Adjust Pattern to 
Sensor Failure 


.0447 


-.5791 


.1632 


Extend Sensor Pattern 


.1172 


-.5526 


-.5211 


Anticpate Target 
Movement 


-.2909 


.7820 


-.2064 


Construct Sensor 
Monitoring Pattern 


.1720 


-.6017 


-.2467 


Coordinate Hand-Off 


1.4256 


-.4035 


.6366 


Determine Signal Is 
Valid Contact 


-1.3964 


.0029 


.2927 


Determine Weapon 
Setting For Atack 


.6826 


.6964 


.0826 


Gain Attack Criteria 


.4285 


.7912 


-.0787 


Determine Target Fix 


-.5060 


.7070 


-.1407 


Create Sensor pattern 


.2664 


-.5176 


-.5525 


Manage Equipment 
and Stores 


.0169 


-.4266 


.6182 


Classify Signal 


-1.3337 


-.0282 


.3539 


Compensation for 
Propagation Conditions 


-.2605 


-.6237 


-.3427 


Determine Aircraft 
Launch Position 


.6336 


.7694 


-.0585 
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Figure 8. Dimension 1 Unidimensional Plot 
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Figure 9. Dimension 2 Unidimensional plot 
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Figure 10. Dimension 3 Unidimensional Plot 



1. Interpretation of Dimension One: Workload 



This dimension was interpreted as representing the 
workload of the TACCO (physical and cognitive) when 
executing the decision function (Figure 8). The extreme 
points of this mapping - Coordinate Hand-off, Classify 
Signal, and Determine Signal is Valid Contact lend 
reinforcement to this interpretation. Although the TACCO is 
ultimately responsible for the successful completion of the 
tactical mission, the sensor operators (acoustic and 
nonacoustic) are required to Classify Signal and Determine 
Signal is Valid Contact. The TACCO may verify the sensor 
operator's decision but as far as workload of those decision 
functions to the TACCO is concerned, the workload is 
minimal. The workload of these decision functions lie in the 
interpretation and analysis of target signals, which is more 
of a sensor operator's function. On the other extreme is 
perhaps the heaviest workload decision function of the 
decision set - Coordinate Hand-off. Many inputs are required 
to be processed and assimiliated to provide a well executed 
Coordinated Hand-off. Although the format of a Coordinated 
Hand-off is well established, the degree of detail required 
to maintain target contact is great. 

The implicit ordering of decision functions between 
these extreme stimuli support the workload interpretation of 
this dimension. The next three highest decision functions 
after Coordinated Hand-off - Determine Weapon and Setting, 
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Determine Aircraft Weapon Launch Position, and Gain Attack 
Criteria - involve great complexity. Because the actual 
depth of the target is difficult to assess, the TACCO must 
use every input to the problem to determine the correct 
weapon selection and its search setting in order to maximize 
probability of kill. Maneuvering the aircraft into weapons 
launch position and gaining attack criteria must occur 
within a small window in time in order to place a weapon on 
target. A small miscalculation in the execution of the 
decision functions could cause the aircraft to be out of 
position when attack criteria is gained resulting in a lost 
attack opportunity. 

A cluster of five decision functions concerning 
sensor placement, are found close to the zero point - Create 
Sensor Pattern, Construct Sensor Monitoring Pattern, Extend 
Existing Sensor Pattern, Adjust Pattern to Sensor Failure, 
and Manage Equipment/Stores to Accomodate Present and Future 
Needs. All of these decision functions exhibit two levels of 
cognitive workload. The more cognitive level concerns the 
situational inputs of each function - the decisions that 
change with time, both present and future. The less 
cognitive level represents the computer controlled decision 
inputs such as monitoring sequence, search pattern, and 
stores remaining, and the formatted procedures established 
by the On-Scene Commander (such as buoy spacing, type of 
pattern, and buoy channel numbers). 
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The three remaining decision functions - Compensate 
for Acoustic/Atmospheric Propagation Conditions, Anticipate 
Target Movement, and Determine Target Fix require less work 
with respect to the TACCO than the 'zero' cluster but more 
work than the extreme signal processing functions. The 
computer on board the S-3A has a bathythermal and a radar 
range program that takes external inputs to establish 
situational conditions. Anticipating Target Movement can be 
assisted by the prediciton and ranging computer functions 
and fixing information is supplied by the sensor operators 
with confidence levels. 

2. Interpretation of Dimension Two: Time Criticality 

Dimension two was interpreted as representing the 
degree of time criticality of the decision functions. The 
unidimensional representation (see Figure 9) shows three 
well defined groupings of decision functions, helpful in 
analyzing this dimension. The most neutral decision 
functions on the criticality dimension are Deterimine Signal 
is a Valid Contact, amd Classify Signal. These signal 
processing functions are considered somewhat time critical 
in determining the tactics to use against the target but 
less critical concerning the prosecution of the target. 

On the high criticality extreme of the 
representation, the actual tactical decision functions - 
Gain Attack Criteria, Anticipated Targert Movement, 
Determine Aircraft Weapons Launch Position, Determine Target 
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are 



Fix, and Determine Weapon and Setting for Attack - 
closely clustered. The relative differences between the 
decision functions were very small numerically, suggesting 
little difference in the degree of criticality. However, the 
dynamic nature of these decision functions requires 
expeditious scrutiny of all time critical inputs to reach 
the most optimal decision. 

The remaining decision functions were clustered at 
the low criticality extreme of the representation. This 
stimuli subset - Coordinate Hand-off, Manage 
Equipment/Stores to Accomodate Present and Future Needs, 
Create Sensor Pattern, Extend Sensor Pattern, Adjust Sensor 
Pattern to Sensor Failure, Construct Sensor Monitoring 
Pattern, and Compensate for Acoustic/Atmospheric Propagation 
Conditions - is very important in the accomplishment of the 
ASW mission. However, time criticality is not normally a 
factor in making any of these decisions. 

Speed of decision making can be considered 
synomymous with time criticality for this dimensional 
interpretation. Figure 9 illustrates areas requiring fast, 
moderate, and more deliberate decision making. 

3. Interpretation of Dimension Three: Complexity 

Dimension three was interpreted as referring to the 
degree of complexity in the decision function with respect 
to the TACCO. Figure 10 shows a rather scattered 
unidimensional plot of the decision function. Two small 
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clusters on either extreme and an evenly distributed 
grouping of the remaining decision functions about zero made 
this dimension the most difficult to interpret. 

The decision functions on the high extreme of 
Complexity - Coordinate Hand-off, and Manage 
Equipment/Stores to Accomodate Present and Future Needs - 
showed high degrees of decision complexity, assembling 
information for hand-off and inventorying sensors. While on 
the low extreme of the complexity dimension, the stimuli - 
Extend Sensor Pattern and Create Sensor Pattern - required 
little original thought and simply followed standard tactics 
procedures. The complexity of either decision function was 
minimal . 

The remaining decision functions are evenly spaced 
around zero suggesting moderate complexity. The stimuli - 
Classify Signal, and Determine Signal is a Valid Contact, - 
has a greater degree of complexity, processing all inputs to 
those decision functions. On the other end of the grouping. 
Compensate for Acoustic/Atmospheric Propagation Conditions, 
minimizes complexity through computer-assistance programs. 
The rest of the 'zero' grouping involve implicit degrees of 
complexity . 
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E. PRIORITIZATION OF THE DECISION SPACE 

The three dimensions constructed by the MDS analysis of 
the sorting data represent a geometric model of experienced 
S-3A TACCOs' understanding of Air ASW decisions. These three 
dimensions represent the basic implicit relationships that 
TACCOs used in the Air ASW decision making. Therefore, other 
relationships or characteristics should exist only as 
combinations of these underlying features. One such 
characteristic is priority or importance. Two of the rank 
orderings, discussed earlier, had as criterion - importance 
to the mission. This section discusses the implicit 
combination of these three dimensions used by the TACCOs 
when they ranked the 14 decision functions using the 
importance criterion (both with attack and without attack) . 

The basic technique used to determine TACCO priority 
functions is termed Priority Mapping, and has been developed 
directly for this type of application by Zachary [1980b] . It 
is related to the psychometric techniques known as 
Preference Mapping [Carrol and Chang 1967] and is based on 
Coombs [1950] and Bennett and Hayes [1960] notion of 
Unfolding Analysis. , 

1. Unfolding Analysis and Priority Mapping 

In the multidimensional model of the Air ASW 
decision space, each decision function in the stimulus set 
has a unique projection or coordinate on each of the 
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dimensional axes. Each dimension is considered to be a scale 
which measures some relevent perceptual attribute of the 
decision space. Collectively, the three scales in this model 
measure all of the independent features needed to represent 
the perceived interrelationships among decision functions in 
the decision space. In this model, the three dimensions 
provide numerical measures of all the important features on 
which experienced S-3A TACCOs perceive ASW decisions (as 
shown in Table IX) . 

Therefore, other salient characteristics of the 
decision functions must be combinations of the features 
represented by the axes of the MDS model. These derivative 
characteristics or features may have differing measurement 
scales, resulting in different representations within the 
multidimensional space. The measurement scales include 
categorical, ordinal, and interval-ratio. The categorical 
scale partitions the multidimensional space into unordered 
regions. The ordinal scale provides a graded or ordered 
partition of the multidimensional space. Finally, the 
interval-ratio scale represents the feature as a directed 
vector or curve passing through the multidimensional space. 

Unfortunately, little can be done to represent 
categorical derivative features in the multidimensional 
model. However, through the use of Unfolding Analysis, it is 
possible to construct a representation of ordinal or 
interval-ratio derivative features. The technique is based 
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on the concept that the general form of the representation 
of a derivative feature in a predefined multidimensional 
space can be determined by comparing different models of 
increasing complexity [Zachary 1980b] . 

Unfolding Analysis restricts itself to four specific 
formulae which have simple mathematical forms and direct 
psychological interpretations. Although the analysis could 
be performed on either ordinal or interval-ratio scale 
features, the analysis used in this discussion was limited 
to ordinal scales, consistent with the (ordinal) ranking 
data . 

In Unfolding Analysis, the four models are 
considered and compared in terms of their ability to 
represent a given ordinal scale in a multidimensional space. 
Usually, the ordinal scale is a ranking of a stimulus set by 
individuals according to some criterion and the 
multidimensional space is a result of some MDS analysis. 
Unfolding Analysis attempts to represent the ranking 
criteria as a function of the MDS dimensional izat ion . 

Techniques, similar to regression, can be used to 
define an equation which models the criterion representation 
as a function of the MDS dimensions. Carroll and Chang 
[1977] developed a set of computational procedures for 
performing these regression-like analysis. In this analysis, 
their computational approach is used to model data on 
perceived priority of decision functions. This adaptation of 
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since 



the model is appropriately termed 'Priority Mapping', 
it constructs a mathematical 'map' of the TACCOs intuitive 
prioritization of the decision functions. 



2. Four Models of Priority 

The Unfolding Analysis and Priority Mapping 
procedures employ four models of priority. These models are 
based on a heirarchial structure such that each model is a 
special case of the next 'higher' model. Each model in the 
heirarchy subsumes all subordinate models. The models will 
be explained in terms of ordinal representations consistent 
with the data set. 

An ordinal scale in a multidimensional space divides 
the space into graded regions, such that a point in one 
region has a different rank than a point in another region, 
but the same rank as another point in the same region. In 
other words, the space is broken into isopriority regions 
separated by isopriority contours. A geometric 
representation of the four models showing the isopriority 
contour structure is shown in Figure 11. For simplicity. 
Figure 11 and the following discussion are expressed in 
terms of a two-dimensional space. A generalization of this 
approach can be extended to three or more dimensions. 

The first model (and lowest in the heirarchy) is the 
vector model. This model assumes that each dimension 
contributes in a linear fashion resulting in a 
prioritization defined by stimuli projections onto a vector 
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Figure 11. Four Models of Priority 
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in the multidimensional space. The isopriority contours are 
lines (in this case) perpendicular to the stimulus vector 
(shown in Figure 11a) . The vector model is similiar to the 
concept of Multiattribute Utility (MAU) . Each dimension 
(attribute) contributes to the model in a linear way with 
the relative contribution of each dimension defined by a 
coefficient assigned to it. The vector model can be compared 
to the economic's 'more is better' model, because it assumes 
that whatever the dimension, more of it will contribute to a 
higher priority. 

The remaining three models define priority in terms 
of distance in the multidimensional space. These models are 
based on the assumption that there is an ideal point 
somewhere in the modeled space, such that the closer a 
stimulus is to the ideal point, the higher the priority of 
that stimulus. The simplest of the distance models is termed 
the Unweighted Distance Model or just Distance model. This 
model prioritizes stimuli strictly according to their 
distances from the ideal point. This distance measurement 
for this model is Euclidean in nature yeilding isopriority 
contours represented by concentric circles around the ideal 
point (as shown in Figure lib) . The unweighted model assumes 
each dimension has an equal contribution to priority. 

A more general representation in terms of distance 
allows the distance between the ideal point and each 
decision to be a function of weighted dimensionality. More 
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specifically, dimensions may not contribute equally to the 
model so each dimensional axes is given a relative weight. 
This model, termed the Weighted Distance Model, allows the 
isopriority contours to assume the general slope of an 
ellipse parrallel to the dimensional axes as shown in Figure 
11c. 

In the models previously discussed, an assumption is 
that each dimension contributes independently to priority. 
However, interaction may occur among some or all dimensions 
in constituting the concept of priority. To allow such 
interactions among dimensions, a model can be constructed by 
permitting the axes to be rotated before the distances are 
computed. This model, termed the Generalized Distance Model, 
allows each dimension to contribute independently and 
collectively to the priority of an decision function. The 
isopriority contours are similar to the weighted distance 
model except the contours can assume any orientation in the 
space as shown in Figure lid. 

These four models - the vector model, the distance 
model, the weighted model, and the generalized model - are 
the mathematical basis for the Unfolding Analysis and 
Priority Mapping. In Unfolding Analysis, the ability of each 
model to replicate the ordering of the decisions by the 
TACCOs, is evaluated and the most applicable model is 
selected. In Priority Mapping, the specific coefficients of 
the selected model are determined and a precise mathematical 
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prepresentation of priority is constructed. 



F. UNFOLDING ANALYSIS AND PRIORITY MAPPING OF DECISION 

FUNCTIONS 

Unfolding Analysis procedures were conducted for the two 
sets of ranking of decision functions by importance. The 
first analysis considered the rankings by importance in the 
ASW mission where the goal was an attack on the submarine. 
The individual rankings are in Appendix D f and the average 
rankings are given in Table V. The second analysis 
considered the rankings by importance to the ASW mission 
where the goal was surveillance of the submarine. The 
average rankings for this criterion is given in Table VI and 
the individual rankings are in Appendix D. The priority 
mapping was conducted with the PREFMAP program developed by 
Carrol and Chang [1967, 1977] , which performs the Unfolding 
Analysis as described in the last section. PREFMAP, like 
MINASSA, is a part of the MDSX package of Multidimensional 
Scaling programs [Coxon et al 1977] available on the Wharton 
School's DEC-10 Computer. PREFMAP performs Unfolding 
Analysis by conducting seperate analysis of each individual 
ranking against the multidimensional configuration of the 
stimulus set provided by the user. 

1 . Prioritization of Decision Function by Importance in 
Mission with Attack 

Using the PREFMAP program, the analysis began by 
constructing trial representations of the rankings criterion 



71 



using each of the four models discussed earlier. These 

representations produced different rankings of the decisions 

functions, which can be compared to the actual TACCO 

rankings to determine the appropriateness of each model. The 

simplest, most reasonable measure of comparison is the 

correlation between the actual rankings and the rankings 

produced by the model. The correlation, when squared, 

becomes the coefficient of variation, r , indicating the 

proportion of variation in the TACCO rankings accounted for 

in the PREFMAP rankings. A F test statistic can be computed 
, 2 

from r and used to test the null hypothesis, H q , that the 
value of r is strictly a random result. The values of r / 
the F ratio, and the levels of significance are given in 
Table X. 



Table X. Significance of Results for Mission with Attack 



MODEL 


2 

r 


F STATISTIC 


REJECTION LEVEL 


GENERALIZED 


.996 


116.8 


<.01 


WEIGHT DIST. 


.982 


62.6 


<.01 


UNWEIGHTED 


.943 


37.4 


<.01 


DISTANCE 

VECTOR 


.939 


51.2 


<.01 



All four models produced a significant (non-chance) 
representation of the TACCO data. The next phase of the 
analysis is to determine which is the best representation. 

The most widely used criterion for selecting among 
models is the goodness of fit measurement between the model 
and the data. The goodness of fit is measured in all four 
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2 9 

models by r - the coefficient of variation. The r^ value 

for all four models is greater than .99, indicating a good 

fit by all models. In fact, each model in the heirarchy 

accounts for more variation than the lower models. This 

reinforcing result is an artifact of the PREFMAP algorithm 

since each heirarchial model increases the complexity of the 

equation with increases in the number of parameters. 

Consequently, the model with more parameters will explain 

more of the variance than the subordinate models. Therefore, 

it is reasonable to assume that r will increase as the 

heirarchy is traversed from the vector to the generalized 

model. The fundamental question is whether or not the 

increase in r , as the model heirarchy is traversed, is 

significant as the number of model parameters increase. 

To answer this question, a statistical comparison 

among the model rankings was used. A F statistic was 

2 

computed from the r values of each pair of models (the 
between-model coefficient of variation) to test the null 
hypothesis that the increase in the value of r as the model 
heirarchy is traversed is only a chance result. A failure of 
this hypothsis for a given pair of models indicated that 
there was not a significant difference in the explanatory 
power of the two models and the subordinate model would be 
preferred because it is less complex. The comparison among 
the four models is shown in Table XI. 
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Table XI. Comparison of Coefficients of Variation for 

Mission with Attack 



MODEL 



F STAT 

SIGN. 



WEIGHTED 


5.1 








.022 






UNWEIGHTED 


11.2 


7.4 






.018 


.019 




VECTOR 


10.1 


5.5 


21.0 




.021 


.030 


.424 



GENERALIZED WEIGHTED UNWEIGHTED 

As illustrated in Table XI, the generalized 
distance model is significantly more powerful than the other 
models with a confidence of greater than 95 percent. 
Therefore, the generalized distance model was selected to 
represent the TACCO rankings according to this criterion. 

Having selected the 'best' model for representing 
the ranking criterion, a precise priority function was 
derived using the PREFMAP program. The importance of 
priority of a decision function j in a mission with attack 
(P wa (dj)) was given by: 

P wa (dj) = -,148x 2 jl -1 . 74x 2 j2 -. 417x 2 j3 



+ . 330Xj^Xj2 • "7 7 4 x j ^ x j ^ 1 • 20x j 2^ j ^ 

+ .018Xj 1 -. 091Xj 2 -. 670Xj 3 +.727 



where Xj^ 


is 


the 


coordinate 


of dj 


on 


dimension 


1 


X j 2 


is 


the 


coordinate 


of d j 


on 


dimension 


2 and 


X j 3 


is 


the 


coordinate 


of dj 


on 


dimension 


3. 


The 


values of Xj^, 


x j2' x 


j 3 


are given 


in Table IX. 
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The priority values of the fourteen decision function 
generated by this function are given in Table XII. 



Table XII. Decision Function Prioritization for 
Mission with Attack 



PRIORITY 

RANK 


NAME 


PRIORITY 

SCORE 


1 


GAIN ATTACK CRITERIA (AC) 


-.2489 


2 


CLASSIFY SIGNAL (CS) 


-.2464 


3 


DETERMINE SIGNAL IS VALID 






CONTACT (VC) 


-.1947 


4 


DETERMINE AIRCRAFT WEAPON 






LAUNCH POSITION (LP) 


-.1852 


5 


DETERMINE WEAPON AND SETTING 






FOR ATTACK (DS) 


-.1785 


6 


ANTICIPATE TARGET MOVEMENT (TM) 


-.1431 


7 


DETERMINE TARGET FIX (TF) 


-.1198 


8 


CREATE SENSOR PATTERN (CP) 


.0402 


9 


CREATE SENSORING MONITORING 






PATTERN (MP) 


.0444 


10 


EXTEND SENSOR PATTERN (EP) 


.0673 


11 


COMPENSATE FOR PROPAGNATION 






CONDITIONS (PC) 


.1368 


12 


ADJUST PATTERN TO SENSOR 






FAILURE (AP) 


.1867 


13 


MANAGE EQUIPMENT AND STORES (ME) 


.1980 


14 


COORDINATE HAND-OFF (CH) 


.3656 



These values represent the relative priority of the 
decision functions in a mission with attack. The lower 
values (i.e., negative) in Table XII indicate a higher 
priority and the higher values represent a lower priority. 
This ordering is the result of the TACCOs assigning lower 
ranks to the more important decisions and higher ranks to 
the less important decisions (PREFMAP program merely 
maintains this directionality in its computations) . 
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2. Prioritization of Decision Function by Importance in 
Mission Without Attack 

An identical analysis was conducted for the TACCO 

rankings of decision functions where the criterion was 

importance to ASW mission without attack. PREFMAP 

constructed the initial representations to determine which 

of the four models produced 'best' representation of the 

2 

TACCO data. The coefficient of variation (r ) and the 
statistical test (F) (Table XIII) were used to determine 
which model is the best representation. 



Table XIII. Significance of Result for Mission 

Without Attack 



MODEL 


r 2 


F STATISTIC 


REJECTION LEVEL 


GENERALIZED 


.997 


130.9 


<.01 


WEIGHT DIST. 


.949 


21.8 


<.01 


UNWEIGHTED 


.881 


16.7 


<.01 


DISTANCE 

VECTOR 


.795 


13.0 


<.01 



As with the previous criterion, the null hypothesis 
can be rejected for all four models, indicating that a 
significant representation of the TACCO ranks were provided 
by all models. 

As before, the four models were compared against 
each other to determine the model of best representation. 
The 'between model' F statistic was used, testing the null 
hypothesis, H q , that the increase in the proportion of 
variance accounted for by the more powerful model was a 
chance result accounted for by its increased number of 
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parameters. Table XIV show the 1 between-model ' F statistic 
and the associated significance level. 



Table XIV. 
Mnnrr 


Comparison of 
for Mission 


Coefficients of 
Without Attack 


Variation 


rlU UCj Li 




F STAT 


SIGN. 


WEIGHTED 


18.7 

.008 






UNWEIGHTED 


27.3 

.003 


4.7 

.051 




VECTOR 


39.6 

.002 


7.0 

.016 


6.5 

.031 



GENERALIZED WEIGHTED UNWEIGHTED 



There were more pronounced differences in the 
significance levels between models in Table XIV than in 
Table XI. More specifically, the Generalized Distance Model 
was found to be significantly better in representing the 
TACCO data with a confidence of greater than 99 percent. As 
a result, the Generalized Distance Model was selected as the 
most appropriate model for the no attack criterion. 

As in the previous analysis, the PREFMAP program was 
used to 'map' the precise mathematical priority function. 
Using this algorithm, the importance of decision j on a 
mission without attack ( p woa (dj)) was given by: 

P woa< d j> * -•230x 2 jl -1.93x 2 j2 -,815x 2 j3 

+.623Xj^Xj2 + *385Xj^Xj2 -.592Xj2*j3 

+.226Xj 1 +.260Xj 2 -.410Xj 3 +.888 
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where x^ is the coordinate of decision j on dimension 1 

X j2 coor dinate of decision j on dimension 2 and 

Xj^ is the coordinate of decision j on dimension 3. 

The values of the Xj^, as before, were given in 

Table IX. The priority values for the fourteen decision 
functions are given in Table XV. 



Table XV. Decision Function Prioritization for Mission 

Without Attack 



PRIORITY 

RANK 


NAME 


PRIORITY 

SCORE 


1 


CLASSIFY SIGNAL (CS) 


-.2309 


2 


DETERMINE SIGNAL IS VALID 






CONTACT (VC) 


-.2210 


3 


DETERMINE TARGET FIX (TF) 


-. 1612 


4 


ANTICIPATE TARGET MOVEMENT 


-.1473 


5 


COORDINATE HAND-OFF (CH) 


-.1246 


6 


EXTEND SENSOR PATTERN (EP) 


-.0637 


7 


CREATE SENSOR PATTERN (CP) 


-.0539 


8 


CONSTRUCT SENSOR MONITORING 






PATTERN (MP) 


-.0523 


9 


COMPENSATE FOR PROPAGATION 






CONDITIONS (PC) 


-.0456 


10 


MANAGE EQUIPMENT AND STORES (ME) 


.0203 


11 


ADJUST PATTERN TO SENSOR 






FAILURE (AP) 


.0539 


12 


GAIN ATTACK CRITERION (AC) 


.2025 


13 


DETERMINE AIRCRAFT WEAPON 






LAUNCH POSITION (LP) 


.3563 


14 


DETERMINE WEAPON AND SETTING 






FOR ATTACK (DS) 


.4245 



These values represent the prioritization of the 
decirsion functions in the ASW mission without attack on the 
submarine. For reasons discussed in the previous section, 
the smaller values represent higher priority in Table XV. 
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V. DISCUSSION 



A. RESULTS 

1. Summary 

In this thesis, a methodology was developed that 
prioritized ASW decision functions according to different 
criteria. However, to more fully understand the results of 
the methodology and its components, the purpose of this 
thesis should be reviewed. The objective of this effort was 
to examine ASW decision making in the S-3A through the use 
of mathematical modeling techniques and to establish a 
prioritization technique for the development of decision 
aids to assist decision making in the system. An examination 
of the S-3A ASW decision making included discussions of the 
history of ASW, the four and six partition spacing of the 
Air ASW mission, and finally the fourteen decision function 
partition of the Air ASW decision space. S-3A TACCOS sorted 
and ranked the 14 decision functions through the use of the 
mathematical modeling techniques of Multidimensional Scaling 
and Unfolding Analysis, a priority mapping of the fourteen 
decision functions was produced. 

The Multidimensional Scaling representation of the 
decision functions was determined to have three orthogonal 
dimensions or axes through a goodness of fit test. The 
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dimensions were interpreted as follows: 

Dimension One - Workload 
Dimension Two - Time Criticality 
Dimension Three - Complexity 
It should be noted that this interpretation of 
dimensional meaning is not unique. The interpretations, to a 
great degree, are subjective in nature relying upon the 
analyst's understanding of the interrelationships of the 
stimulus set. However, the relative positions of the stimuli 
on each dimension (Figures 8 through 10) is unique so the 
interpretations of the dimensions should be roughly 
equivalent from analyst to analyst. 

The Unfolding Analysis determined that the 
importance rankings were best represented by the Generalized 
Distance Model in terms of priority functions. Only two of 
these four rankings were utilized in the unfolding analysis. 
The criterion of importance was assumed to be the most 
important feature of the priority mapping, so the criteria 
of importance to the ASW mission with attack and importance 



to the ASW 


mission 


without attack 


were selected 


and 


the 


criteria of 


urgency 


and workload 


rankings were 


used 


to 



validate the resulting Priority Mapping algorithm. 

Finally, the results of the Multidimensional Scaling 
program (MINASSA) and the Unfolding Analysis program 
(PREFMAP) were used to 'map' the precise priority function. 
From the two priority functions, a prioritization of the ASW 
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decision space was established according to both criteria 
(see Tables XIII and XV). 



As a result, a priort i za t ion technique for Air ASW 
decision functions was established for the development of 
decision aids in the S-3A. The fundamental question raised 
by the priority ranking shown in Tables XIII and XV is what 
is the meaning of this prioritization of decision functions. 

2. Interpretation of Results 

To better interpret the results of the priority 
mapping function, a relative plot of the 14 decision 
functions is given in Figure 12 for the attack and no-attack 
criterion. The relative position of the decision functions 
on this priority scale for each criterion yields an implicit 
ordering of areas in which TACCOs think decision aids would 
be useful. 

a. Gain Attack Criteria 

The Gain Attack Criteria decision function was 
ranked first in the priority scale for the 'attack' 
criterion. Intuitively, this result is not surprising in a 
mission where the goal is attacking the submarine. The 
TACCOs interviewed felt that in a mission where attack on a 
hostile threat was the goal, if attack criteria was not 
gained, the mission would be a failure. 

The implication, from the priority scale of the 
importance to the ASW mission with attack, is that more 
energy should be channeled into the area of decision aids 
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IMPORTANCE 

W/ ATTACK W/OUT ATTACK 




KEY 

AP - ADJUST PATTERN TO SENSOR FAILURE 

EP - EXTEND SENSOR PATTERN 

AM - ANTICIPATE TARGET MOVEMENT 

MP - CONSTRUCT SENSOR MONITORING PATTERN 

CH - COORDINATE HAND-OFF 

VC - DETERMINE SIGNAL IS VALID CONTACT 

DS - DETERMINE WEAPON AND SETTING FOR ATTACK 

AC - GAIN ATTACK CRITERIA 

IF - DETERMINE TARGET FIX 

CP - CREATE SENSOR PATTERN 

ME - MANAGE EQUIPMENT/STORES 

CS - CLASSIFY SIGNAL 

PC - COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 
LP - DETERMINE AIRCRAFT WEAPON LAUNCH POISITCN 



Figure 12. Priority Score of Decision 
Functions (S-3A) 
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concerned with assisting the TACCO in determining when 
attack criteria has been reached. Additionally, Gain Attack 
Criteria ranked highest in the average rankings for 
importance to ASW mission with attack (Table V) and Workload 
(Table VIII) and second highest in the ranking for the 
Uregency (Table VII) . This placement distinguishes this 
decision function as time-critical for the TACCO. 

Placement of the Gain Attack Criteria decision 
function in the MDS solution space (Figures 5 through 10) 
also confirms this distinction. Although this decision 
function is positioned near the middle of dimension three 
(Complexity) , it is located at the positive extreme of both 
dimension one (Workload) and two (Time Criticality) . This 
positioning indicates a highly time critical and work 
intensive decision function. Based upon this analysis, the 
decision function - Gain Attack Criteria - represents an 
area in the Air ASW mission where implementation of a 
decision aid would be of great benefit. 

b. Classify Signal and Determine Signal is Valid 
Contact 

The decision functions - Classify Signal and 
Determine Signal is Valid Contact - were ranked first and 
second in the priority scale for the 'no attack' criterion 
(Figure 12) and ranked second and third on the priority 
scale for the 'attack' criterion (behind Gain Attack 
Criteria). This position on the high extreme of both 
priority scales was unexpected however it was clear that the 
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TACCOs felt these two decision functions to be the heart of 
a successful ASW mission. The TACCOs felt that the entire 
mission was dependent on the proper evaluation of an 
incoming signal (that could be a hostile threat) and 
suggested that because this evaluation is so important, 
there is little room for error regarding either decision 
function . 



The placement of these two decision functions 
(Classify Signal and Determine Signal is Valid Contact) on 
the high extreme of both priority scales suggest that these 
two areas in the Air ASW mission are most important in the 
accomplishment of the mission. The implication is that 
decision aids should be developed for both decision 
functions. Furthermore, both decision functions were ranked 
high in the average rankings for importance to mission with 
attack and without attack. In the average ranking for 
Urgency, both decision functions were positioned in the 
center of the ranking, indicating that Classify Signal and 
Determine Signal is Valid Contact are not critical in terms 
of Urgency. The placement of the decision functions at the 
bottom of the average ranking for Workload indicates that 
both decision functions do not add to the TACCO Workload. In 
the S-3A, Classify Signal and Determine Signal is Valid 
Contact are responsibilities of the TACCO but these decision 
functions are normally delegated to the sensor operators 
with inputs to the TACCO. 
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The two decision functions - Classify Signal and 
Determine Signal is Valid contact - in the sample of the 
fourteen lay at opposite extremes in two dimensions and in 
the center of the third of the MDS space (Figures 5 through 
10). In dimension one. Workload, Classify Signal and 
Determine Signal is Valid Contact lies at the very negative 
extreme representing light workload (Figure 8). 

On the second dimension. Time Criticality, the 
two decision functions are clustered at the 'zero' point, 
well away from the other two well-defined groupings 
(Figure 9). This result suggest that Classify Signal and 
Determine Signal is Valid Contact are not as time critical 
as the 'attack' functions because the confidence in the 
Contact Validity normally improves over time. However, both 
decision functions are more time critical than the 'tactics' 
functions because confidence improves only after contact is 
established . 

The third dimension, complexity, located both 
decision functions in the positive region indicating a high 
degree of complexity (Figure 10) . Many inputs into the 
classification or verification decisions from all sensors 
suggest that the degree of complexity increases with the 
number of sensors employed. Classify Signal and Determine 
Signal is Valid is a sine quad non . 

The remaining eleven decision functions were not 
addressed further because the apparent order of the three 
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discussed decision functions is not implicitly present in 
the ordering of the last eleven functions. This problem will 
be discussed further in the Conclusion and Recommendations 
sections . 

3. Comparision of Results to Similiar Work 

This analysis has established priority scales for 
both attack and no attack decision situations. The fourteen 
decision functions were ordered in importance by the use of 
Multidimensional Scaling combined with Unfolding Analysis. 

Zachary (1980 a and b) used the same prioritization 
methodology used in this analysis in conjunction with P-3C 
TACCO inputs, the results of his analysis was very similar 
to this analytical result. Although the P-3C is a shore 
based ASW aircraft with a crew of 11, the function of the 
TACCO is very much like that of the S-3A TACCO. 
Consequently, similiar results in the prioritization 
analysis would not be surprising. 

Zachary's prioritization for Importance to the ASW 
Mission with Attack on the Submarine was very nearly the 
same as the S-3A prioritization. These were minor inversions 
of one or two decision functions in the final priority scale 
comparisions , however the differences in the aircraft system 
capabilities suggest that this result is reasonable. The 
comparision of decision function prioritization for mission 
without attack on submarine provided a similiar result. The 
priority scales were very similiar except for the inversion 
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of one or two decision functions due to aircraft and 
tactical dissimiliarities . Figure 13 shows Zachary's 
priority scale for both attack and no-attack for the P-3C. 
These priority scales are presented in the same format as 
Figure 12 for ease of comparison. 

Zachary's analysis also used three dimensions as 
representative of the MDS decision space (discussed in 

Chapter Four). Zachary's interpretation of his three 
dimensions for the P-3C data was similiar to this analysis, 
however, the placement of each decision function in the 
decision space was markedly different. His dimensional plots 
can be found in Appendix E. This difference could be 

contributed to sampling differences, aircraft differences, 
or training differences between S-3A and P-3C TACCOs . 

Zachary's results reinforces the applicability of 
the results of this analysis. Although two analyses of 
similiar data do not necessarily validate a new methodology, 
the degree of agreement between results suggest * further 
application could be of great benefit in the area of ASW. 



B. CONCLUSION 

1 

This thesis has established that a workable mathematical 
technique exist for the prioritization of Air ASW decision 
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IMPORTANCE 




AP - ADJUST PATTERN TO SENSOR FAILURE 

EP - EXTEND SENSOR PATTERN 

AM - ANTICIPATE TARGET MOVEMENT 

MP - CONSTRUCT SENSOR MONITORING PATTERN 

CH - COORDINATE HAND-OFF 

VC - DETERMINE SIGfiAL IS VALID CONTACT 

EE - DETERMINE WEAPON AND SETTING FOR ATTACK 

AC - GAIN ATTACK CRITERIA 

TF - DETERMINE TARGET FIX 

CP - CREATE SENSOR PATTERN 

ME - MANAGE EQUIPMENT/STORES 

CS - CLASSIFY SIGNAL 

PC - COMPENSATE FOR IN-SITU PROPAGATION CONDITIONS 
LP - DETERMINE AIRCRAFT WEAPON LAUNCH POISITCN 

Figure 13. Priority Score of Decision 
Functions (P-3C) 
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functions, specifically and a general application could be 
established for any complex decision space. The 
prioritization methodology developed in this thesis 
represents a new approach to the problem of assigning 
priorities to decision stimuli. This methodology is based on 
two assumptions: 

1) the decision space is a multidimensional domain 
where all underlying dimensions have some 
potential relevancy to the final prioritization. 

2) inputs to the decision space must be based on 
experience, judgement and intuition. 

The prioritization methodology. Priority Mapping, 
produced two siginificant results. First, the MDS portion of 
the methodology identified the three dimensions which 
underlie the TACCO decisions. Secondly, the 'marriage' of 
the MDS results and the Unfolding Analysis provided a 
numerical prioritization of the Air ASW decision space. 

The Priority Mapping algorithm has several interesting 
characteristics. First, the sorting and ranking positions of 
the data collection is only required once. Repeated testing 
or interviewing of the same subjects is not necessary as in 
many consensus techniques such as Delphi. Ease of data 
collection is a definite advantage of this technique. 
Second, specific as well as general questions can be 
answered by reweighting the parameters of the MDS portion of 
the algorithm (as discussed in Chapter IV). Third, an 
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advantage of this algorithm is the built-in cross checks and 
validations carried out by the interaction of the average 
rankings, MDS unidemnsional plots, and the priority scales. 
In other words, placement of a particular stimulus on the 
priority scale should be reflected in that stimulus' 
placement on the MDS plots and in the average rankings of 
the test criterion. Additionally, the priority scale and 
average ranking place would have a positive influence on the 
analyst's interpretation of the MDS dimensions. Fourth, the 
Priority Mapping algorithm provides more information about 
the stimulus set than a ranking or an MDS dimensional 
analysis alone. To achieve similar results using only 
ranking task or sorting task, the subjects would have to 
agree completely on the placement of each stimuli. The 
algorithm eliminates the need for concensus by combining the 
two techniques into a predictor model. 

The operational application of the priority mapping 
algorithm to fleet assets is where the developmental 
'pay-off' is received. This technique has advantages that 
unit commanders can appreciate: 

1) ease of data collection - simple forms with 
simple instructions. Any sailor could administer 
the interview with favorable results. 

2) speed of data collection - fast with little 
confusion. The unconstructed sorting and rankings 
for a large stimulus set would take less time than 
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other similiar techniques (i.e. Pairwise MDS) with 
much smaller stimulus sets. 

3) revisit requirements - one data collection visit 
to operationsl units. The unit commander is not 
compelled to provide subjects for data collection 
or verification time after time. 

4) tangible results - a priority scale of the 
stimulus set. A priority scale of the stimulus set 
is tangible results of the data collection, 
providing real answers to operational problems. 

Many of the techniques discussed in Chapter II do not 
exhibit these advantages. Fleet acceptance of problem 
solving methodologies is essential to the development of new 
software and hardware to meet any present or future threat. 

This prioritization methodology shown in Figure 3 can be 
adjusted to fit any complex decision space provided the 
assumptions of dimensional relevency and judgmental inputs 
are not violated. The methodology is flexible enough to be 
used by any branch of the Armed Forces or any sector of 
industry. Areas of time critical, work intensive, 
multi-faceted decision situations are ideal targets for the 
implementation of the prioritization methodology. The 
implementation of this technique will identify decision 
situations where decision aids would be most beneficial. 
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C. RECOMMENDATIONS 



The decision functions - Classify Signal and Determine 
Signal is Valid Contact - are at the top of the priority 
scale of the 14 decision functions defining the Air ASW 
decision space for the S-3A. Both decision functions were 
the highest priority in the no-attack mission and second and 
third highest priority in the attack mission. Therfore, it 
is recommended that a decision aid be developed to assist 
the S-3A TACCO in the execution of the decision funcitons - 
Classify Signal and Determine Signal is Valid Contact. 

Although the ASW decision space has been prioritized for 
the S-3A, the explicit priority ordering of Classify Signal 
and Determine Signal is Valid Contact across both priority 
scales is not readily apparent in the remaining decision 
function. In other words, it is difficult to assess which of 
the remaining 12 members of the S-3A ASW decision space 
would be next in priority across both priority scales. 
However, an algorithm for combining the relative weights of 
each priority scale across all decision functions such as 
the Mission Operability Assessment Technique [Helm and 
Donnell, 1978] would solve the problem by providing one 
priority scale across all criterion. 

Therefore, it is recommended that an algorithm be 
developed that would combine priority scalings over all 
relevant criterion to achieve a combinational prioritization 
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of the decision space that would be more useful in 
development of decision aids. 



the 
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APPENDIX A 



TEXT OF INTRODUCTORY BRIEFING GIVEN TO S-3A TA£COS PRIOR 
TO MULTIDIMENSIONAL SCALING INTERVIEWS 

I would like to begin by thanking you all for coming 
here today to help us in this effort. You've been asked to 
come here because of your familiarity with ASW and ASW 
decision making. Before we ask you some specific questions 
about ASW , I'd like to begin by providing a brief background 
on what it is that we mean by decision aids and decision 
aiding, what the overall structure of our program is, where 
we have been, and why we have come here to talk to you. As 
good a place as any to begin is with the definition of a 
decision aid. 

Very simply put, a decision aid is any kind of device 
that helps humans make better, more efficient, clearer, and 
faster decisions. Now, obviously, a wide range of possible 
things can be considered decision aids - from a pencil and a 
sheet of paper which enable you to do calculations to large 
computer systems and programs. What we're most interested in 
are specific tools that will enable you as a TACCO to 
interact with your on-board computer system to help you the 
kinds of decisions that you have to make in the course of an 
ASW mission. 

The Navy's interest in decision aiding has been 
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increasing significantly recently because of the realization 
that warfare, in general, and ASW, in particular, is 
becoming more automated and more highly technological. The 
speed and complexity with which decisions must be made is 
increasing constantly to a point where, in the not too 
distant future, you as TACCOs, will be overloaded, possibly 
beyond your capability to make necessary decisions within a 
reasonable amount of time. Therefore, the Navy is interested 
in developing computerized systems of decision aids that 
will help you keep pace with the increasing automation 
on-board your aircraft. It should be pointed out that these 
decision aids will not take you out of the decision making 
process or automate your functions. Rather, they will 
provide you with better, more intelligent support from 
machines and will give you time to do what you do best - 
think and make decision. The whole concept of decision 
aiding is based on the notion that the most complex, useful, 
and important piece of equipment on any platform is the 
human-brain. Humans are on-board to make decisions; but 
computers and other kinds of devices can assist by managing 
information, making certain kinds of data available at your 
fingertips, helping you remember things (like the procedures 
you must go through to accomplish a specific function), 
performing certain kinds of calculations for you, and so 
forth . 

Decision aids can work at a variety of levels. Simple 
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decision aids can, for example, provide very rudimentary 
bookkeeping functions or provide checklists of things that 
must be done. More complex decision aids can "ask" you the 
questions that you should be asking yourself. Still more 
complex decision aids can anticipate some of the information 
that you may want, based on your past performance; they can 
perform certain kinds of calculations automatically so that 
the results of the calculations will be available when 
requested; they can make certain kinds of inferences about 
what might happen in the future enabling you to play "what 
if" games to find out what results might be obtained if a 
specific course of action is undertaken. These latter 
things, which are at the higher end of the spectrum of the 
capabilities of decision aids, are the ones that we're most 
interested in. 

To give you a feeling for the types of aids that are 
possible, I'd like to review some of the other decision aid 
projects of this type that are on going today in the Navy. 
The largest decision aiding effort to date has been 
undertaken by the Office of Naval Research (ONR) . Their 
program has concentrated on developing a variety of decision 
aids for carrier-based air strike operations. Some of the 
specific problems to be addressed by these decision aids 
are; planning the ingress route for an incoming 
carrier-based air strike or reconnaissance mission through a 
complex sensor field; determining the specific timing of 
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alpha type air strikes; task force EMCOM planning; and 

overall air strike campaign planning. The decision aids 

\ 

being developed by ONR are very high level decision aids, 
both in the sense that they provide a great deal of 
assistance to the human and in the sense that they deal with 
very high level command and control decisions (at the task 
force commander level or higher) . 

More germaine to our discussion today is the effort that 
we have undertaken under the joint sponsorship of the Naval 
Air Development Center and the Office of Naval Research to 
identify possible decision aids for on-platform ASW 
operations. I'd like to review some of the earlier parts of 
this effort to clarify why we are here today. 

We began our effort by looking at the ASW platforms that 
will be in use in the 1980-1985 timeframe - the P-3C, the 
S-3A , and the LAMPS MK-III. We examined the specific 
missions that are undertaken by these three platforms in 
order to define a generic or generalized ASW mission, to 
identify commonalities in the missions flown by the three 
platforms, and to identify some of the critical difference 
both in crew functions and in the details of the missions 
that were undertaken. We then constructed a flowchart of the 
sequence of operations that takes place in the generalized 
mission. The mission was subdivided into very broad 
categories - movement to the search area, on-station search, 
prosecution of the contract, possibly culminating in attack 
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and destruction of a hostile submarine. These states were 
then subdivided into the detailed steps that involved 
specific decision about classification and sensor extension 
of search area, and anticipation of target movement. We 
identified the specific sequences in which these decisions 
were made, recognizing that the interrelations between these 
decisions are highly dependent on the sequential nature of 
the ASW mission. The decisions that take place in the attack 
phase, for example, are dependent upon the succesful 
completion of those portions of the mission that relate to 
search and early prosecution of a contact. Ultimately, we 
identified six broad areas that we termed decision making 
situations. We defined a decision making situation as a 
portion of a mission in which complex sequence of decisions 
has to be made by the TACCO. These situations were complex 
because they involved trading off a number of factors 
against one another. The six decision situations were: 

1) On-Station Search, 

2) Contact Classification/Verification 

3) Target Localization, 

4) Surveillance Tracking, 

5) Lost Contact Reacquisition, and 

6) Attack Planning. 

The next issue we addressed in our program was the 
relative priority of each of these situations. By 
prioritizing the decision situations, we felt that we were 
immediately confronted by the problem that there was no 
single dimension, or criteria, by which we could prioritize 
the decision situations. They obviously are influenced by 
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their sequence in the mission. They are also influenced by 
varying time constraints on the decisions that must be made 
in each situation and how busy the TACCO is in each 
situation. The more we thought about it, the more different 
criteria for prioritization we were able to define. It 
became clear to us that one of the problems was that we, as 
analysts, could not determine the prioritization. We decided 
that the only way to determine the specific criteria that 
were relevant to the prioritization of these decisions was 
to ask the people who made these kinds of decisions, people 
such as yourselves. That is why we are here today. We want 
to determine how the various decisions that you, as TACCOs 
make, should be pri toritized. To do this, we have to 
determine the dimensions or criteria by which these 
decisions are interrelated in your minds. Then we have to 
determine the importance or priority of these decisions in a 
mission . 

There are a number of techniques that can be used to do 
this. One way would be to ask you, in very lengthy and 
detailed discussion, to try to identify the dimensions which 
are salient to ASW decision making for you. But besides 
taking a lot of your time, it is not clear that the 
technique would work. People are often very unclear about 
the underlying principles they use to think about common, 
everyday things, like decision making. In addition, we 
would have the problem of resolving the differences we 
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encountered between the various people we talked to. So 
instead, we have decided to use a more formal mathematical 
technique known as multidimensional scaling which will take 
less of your time and will enable us to determine both the 
dimensions and the relative importance of the various 
decisions from the same set of data. Multidimensional 
scaling uses the computer program to calculate the 
dimensions from very simple judgements made by you about the 
basic similarity or difference among these various 
decisions. We also decided that we wanted to address not 
just the broad analytic catefories that we have called 
decision situations, but more precise, meaningful, specific 
decisions that are made by TACCOs. We identified 14 of these 
decisions, many of which occur in several of the decision 
situations. There are, of course, many more decisions that 
are made in the course if a mission but the 14 decisions 
that we chose were ones that appeared in more than a single 
decision situation or ones that seemed particulary amenable 
to decision aiding. 

You have in front of you a set of cards. ^ Each card 
describes one of these decisions. We're going to ask you to 
make judgements about which decisions you feel are similar 
or dissimilar, and to rank them by various criteria. The 
results of these judged simil iar i ties and rankings will be 

^Card-packs describing the 14 decision functions passed out 
to interviewees at this point. 
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used by the multidimensional scaling process to 
mathematically determine a set of relationships between 
these decisions that will help us in prioritizing them. It 
will also help us to understand the kinds of distinctions 
that you find most relevent among these various decisions. 
But, most importantly, we feel it will enable us to relate 
our analysis in a concrete way to your knowledge, 
experience, and intuition of the ASW missions. With your 
help, we will be able to determine the best places to apply 
decision aiding techniques in the ASW mission. 

I'd like to add one other note about the way in which 
you should sort and rank these decision functions in later 
portions of this procedure. Do not sort or rank the 
decisions in terms of how you think a decision aid could 
help you make these decision nor in terms of how you think 
they could be made better, but rather in terms of how you 
currently go about making these decisions and how they 
currently are handled on the platforms on which you have 
worked . 
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APPENDIX B 



RESPONSE FORMS FOR INTERVIEWS 

This appendix contains the forms on which the TACCO's 
interviewed at NAS Cecil Field recorded the results of the 
unconstrained sorting and four rankings. All of the pages in 
the appendix were given to each interviewee as a stapled 
packet. The first page was used to record some general 
biographical information on the respondent. The next two 
pages provided the instructions and response sheet for the 
unconstrained sorting of the ASW decisions. The remaining 
pages provided the instructions and response forms for four 
different rankings of the fourteen decisions. 
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BIOGRAPHICAL INFORMATION 



Aircraft Type: 

Rank/Designator: 

Organization: 

Date TACCO Designation: 

Date Mission Commander (if applicable): 

Deployment Locations as TACCO (and dates) : 



103 



INSTRUCTIONS FOR THE 'SORTING" TASK 



You have been given a pack of fourteen yellow caards. On 
each of these cards is a decision or problem that is 
encountered in an ASW mission. You have all probably faced 
these problems many times in your experience as TACCOs. Each 
decision is in some way different from all the others, but 
each decision is not totally unique; some of the decisions 
are more alike than others. What we would like you to do is 
arrange these decisions and problems into groups according 
to how similar they are. That is, if these are a number of 
cards which represent decisions or problems you feel that, 
based on your experience, are similar, then place all these 
cards together. If there is a card which you think is 
sufficiently unique that it isn't similar to any of the 
others, then place it in a group by itself. There is no 
limit on the number of groups you can make or on the number 
of cards you can place in each group. While the final 
definition of what constitutes similar decisions is left to 
you, we would like you to think of it as referring to 
decisions or problems which somehow solve in the same way or 
which place similar demands on you as TACCO. 
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INSTRUCTIONS : Record each group of decisions you have formed 



on a separate block below. Take one group and write the code 
for the decisions in it on the blank line provided. Then do 
the same for another group in the next block until each 
group has been recorded in a separate block. If you wish, 
you may also include a short phrase describing the 
similarity you saw in that group of decisions. 

GROUP 



GROUP 



GROUP 



GROUP 



GROUP 



GROUP 



GROUP 
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INSTRUCTIONS FOR RANKING 1 



We would like you to rank these decision problems in the 
order in which a less-than-optimal decision would have the 
most detrimental effect on the mission. The decision problem 
for which a less-than-optimal decision would have the least 
detrimental impact on the mission should be ranked last, and 
the decision problem for which a less-than-optimal decision 
would have the greatest detrimental impact on the mission 
should be ranked first. For the mission, assume that it is 
an ASW mission in which the submarine is to be attacked and 
destroyed if possible. 
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INSTRUCTIONS: Enter the two-letter codes for the decisions 



below in the order in which you ranked them. That is, write, 
the code for the decision you ranked first next to 'l.*, for 
the decision you ranked second next to '2.' and so on. 



1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 

8 . 

9 . 

10 . 
11 . 
12 . 

13 . 

14 . 
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INSTRUCTIONS FOR RANKING 2 



Rank the decisions according to the same criterion as in 
the previous case, but for the mission, assume as ASW 
mission in which the submarine is only to be tracked and 
handed-off to a relief platform. 
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INSTRUCTIONS: Enter the two-letter codes for the decisions 



below in the order in which you ranked them. That is, 
the code for the decision you ranked first next to 
for the decision you ranked second next to '2.' and so 



1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 

8 . 

9 . 

10 . 

11 . 

12 . 

13 . 

14 . 



write 
’ 1 .' , 
on. 
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INSTRUCTIONS FOR RANKING 3 



Rank the decisions according to their urgency. Define 
urgency as referring to the speed with which the decision 
has to be made once you know it must be made. 
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INSTRUCTIONS : Enter the two-letter codes fir the 
below in the order in which you ranked them. That 
the code for the decision you ranked first next to 
the decision you ranked second next to '2.' and so 



decisions 
is, write 
' 1. ' , for 
on. 



1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 

8 . 

9 . 

10 . 

11 . 

12 . 

13 . 

14 . 



Ill 



INSTRUCTIONS FOR RANKING 4 



Rank these decisions according to your workload during 



each of 


them. 


Consider both your 


cogn 


itive, or mental 


workload 


and 


your 


physical workload 


in ranking 


the 


deicisons 


. Rank 


the 


decision during 


which 


your workload 


is 


heaviest 


first , 


and 


the one during 


which 


your workload 


is 



least heavy last. 
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INSTRUCTIONS: Enter the two-letter codes fir the decisions 



below in the order in which you ranked them. That is, 
the code for the decision you ranked first next to ' 1. 
the decision you ranked second next to '2.' and so on. 



1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 

8 . 

9 . 

10 . 

11 . 

12 . 

13 . 

14 . 



write 
' , for 
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APPENDIX C 



RESULTS OF UNCONSTRAINED SORTINGS OF ASW DECISIONS 

This appendix presents the results of unconstrained 
sortings of fourteen air ASW decision by 30 S-3A Tactical 
Coordinators (TACCO's) stationed at NAS Cecil Field, 
Florida. The sortings were performed as part of interviews 
conducted between 10 December 1979 and 13 December 1979. The 
interview procedure is described in Chapter 3. 

The fourteen decisions that were sorted are shown in 
Table XVI. The instructions for the sorting task, and a 
sample of the form on which the results were recorded, are 
given in Appendix B. 

Each sorting presented as several lists of decisions. In 
this task, each decision function is identified only by a 
two-letter code. The code used for each decision is also 
given in Table XVI. 
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Table XVI. Fourteen ASW Decision Function 
and Alphabetic Codes 



CODE 


DECISION FUNCTION 


AP 


ADJUST PATTERN TO SENSOR FAILURE 


EP 


EXTEND SENSOR PATTERN 


AM 


ANTICIPATE TARGET MOVEMENT 


MP 


CONSTRUCT SENSOR MONITORING PATTERN 


CH 


COORDINATE HAND-OFF 


VC 


DETERMINE SIGNAL IS VALID CONTACT 


DS 


DETERMINE WEAPON AND SETTING FOR ATTACK 


AC 


GAIN ATTACK CRITERIA 


TF 


DETERMINE TARGET FIX 


CP 


CREATE SENSOR PATTERN 


ME 


MANAGE EQUIPMENT/STORES TO ACCOMODATE 
PRESENT AND FUTURE NEEDS 


CS 


CLASSIFY SIGNAL 


PC 


COMPENSATE FOR IN SITU ACOUSTICAL 
AND ATMOSPHERIC PROPAGATION CONDITIONS 


LP 


DETERMINE AIRCRAFT LAUNCH POSITION FOR 
ATTACK ON TARGET 



SUBJECT 1 








GROUP 


DECISION FUNCTIONS 


IN 


GROUP 


1 


ME 






2 


CP,MP,PC,AP,EP 






3 


VC,CS 






4 


TF f AM, AC 






5 


DS, LP 






6 


CH 






SUBJECT 2 








GROUP 


DECISION FUNCTIONS 


IN 


GROUP 


1 


CP, PC, MP, ME 






2 


EP, AP 






3 


VC,CS 






4 


TF, AM 






5 


AC, LP, DS 






6 


CH 
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SUBJECT 3 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP,ME, AP,MP 

EP, PC 

VC,CS 

TF, AM 

AC, DS, LP 

CH 


SUBJECT 4 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP,MP, EP 
PC , ME , AP 
VC ,CS,TF 
AM 

DS, LP, AC 
CH 


SUBJECT 5 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


AM,CP,MP 
PC , AP,ME , EP 
VC , CS, TF 
AC , DS , LP 
CH 


SUBJECT 6 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


CP , PC ,MP, EP 
ME, AP 
VC ,CS ,TF 
CH 

AM,AC,DS, LP 


SUBJECT 7 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 
7 


AM ,CP,MP,EP, DS 
PC 

CS,TF,AP 

ME < 

VC 

AC , LP 
CH 
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SUBJECT 8 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


CP, ME , PC,MP, EP,AP 

TF, AM, LP, AC 

CS,VC 

DS 

CH 


SUBJECT 9 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


PC,MP,CP, AP, EP 

LP,AC,DS 

CS,VC 

DS 

CH 


SUBJECT 10 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


" TF , AM, LP, DS , AC 
PC,VC,CS 
MP,AP,ME 
C P, EP 
CH 


SUBJECT 11 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


C P , M P , E P , PC , AP 

DS , AC , LP 

VC,CS 

TF, AM 

ME 

CH 


SUBJECT 12 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CS,VC 
AC , DS , LP 
MP, PC , ME , AP 
TF , AM 
CH 

CP, EP 
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SUBJECT 13 



GROUP 


DECISION FUNCTIONS 


IN 


GROUP 


1 


CS,VC 






2 


DS,LP,AM,TF,AC 






3 


CH 






4 


AP, EP,MP,PC,CP 






5 


ME 






SUBJECT 14 








GROUP 


DECISION FUNCTIONS 


IN 


GROUP 


1 


AC, AM,DS f TF 






2 


CP, EP, AP 






3 


MP, LP 






4 


VC, PC 






5 


CS 






6 


ME 






7 


CH 






SUBJECT 15 








GROUP 


DECISION FUNCTION 


IN GROUP 


1 


CP, PC,MP,ME, EP,AP 






2 


LP, AM,TF,AC 






3 


CS,VC 






4 


DS 






5 


CH 






SUBJECT 16 








GROUP 


DECISION FUNCTIONS 


IN 


GROUP 


1 


PC,CP,AP,MP,EP 






2 


LP , DS , AC , AM 






3 


CS , VC, TF 






4 


ME 






5 


CH 






SUBJECT 17 








GROUP 


DECISION FUNCTIONS 


IN 


GROUP 


1 


ME , AP, DS ,CH 






2 


PC, CP, AM, TF, AC 






3 


MP, EP 






4 


CS,VC 






5 


CH 
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SUBJECT 18 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


AP,EP,MP,PC,CP 

DS,LP, AM,TF r AC 

CS,VC 

CH 

ME 


SUBJECT 19 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


CP,PC,MP, EP 
AM, AC , DS , LP 
ME,AP 
VC,CS,TF 
CH 


SUBJECT 20 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP,EP 

MP/PC,ME,AP 

AC,DS/LP 

TF,AM 

CS,VC 

CH 


SUBJECT 21 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP,MP,EP,PC,AP 

DS/AC,LP 

TF,AM 

ME 

VC,CS 

CH 


SUBJECT 22 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


PC/ME/AP 

CP,MP,EP 

VC/CS,TF 

DS,LP,AC 

AM 

CH 
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SUBJECT 23 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


AC,DS,LP,AM,TF 

MP,AP,ME 

PC,VC f CS 

CP, EP 

CH 


SUBJECT 24 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


AC/DS/LP 

CP, ME / AP, MP 

EP/PC 

TF,AM 

VC,CS 

CH 


SUBJECT 25 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


PC,MP,CP,EP,AP 

LP,AC,DS 

ME 

VC/CS 

AM,TF 

CH 


SUBJECT 26 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


ME, MP, PC, CP 

AC, LP,DS 

EP,AP 

VC,CS 

TF,AM 

CH 


SUBJECT 27 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 


AP,EP,MP, PC , ME ,CP 

AC, LP, AM, TF 

VC,CS 

DS 

CH 
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SUBJECT 28 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP,MP,PC,EP,AP 
TF, AM, AC 
ME 

CS,VC 
LP, DS 
CH 


SUBJECT 29 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP ,ME , EP, TF , AM , AP, LP, MP 

VC,CS 

PC 

DS 

AC 

CH 


SUBJECT 30 
GROUP 


DECISION FUNCTIONS IN GROUP 


1 

2 

3 

4 

5 

6 


CP ,ME , AP ,MP 

DS , LP, AC 

VC,CS 

TF, AM 

PC , EP 

CH 
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APPENDIX D 



RESULTS OF RANKINGS OF ASW DECISIONS 

This appendix presents the results of rankings of 
fourteen Air ASW decisions (listed in Table XVI) by 30 
TACCOs from NAS Cecil Field. These rankings were performed 
as part of the interview described in Chapter 3. The 
fourteen decisions were ranked according to four different 
criteria. The instructions for these rankings and the forms 
on which the results were recorded are given in Appendix B. 
The first ranking was done according to the perceived 

'importance' of the decision in a mission where the 
objective is to 'attack' and destroy the hostile submarine. 

The second ranking was done according to the perceived 

'importance' of the decision function in a mission where the 
objective is to survey the submarine only. The third rankign 
was done according to the preceived 'urgency' of the 

decisions in whatever type of mission gave then the greatest 
urgency. The fourth ranking was done according to the 
TACCO's perceived 'workload' (both cognitive and physical) 
during each of the fourteen decision functions. 

The results of these four rankings are given below. The 
ranking generates by each individual using each criteria is 
presented as a list of the two-letter codes used in the 
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interviews to represent the decision functions. Table XVI 



(Appendix C) contains the full 
each two-letter code. The order 
listed is arbitrary, but is the 



decision name 
in which the 
same for all 



represented by 
individuals are 
four rankings. 
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RANKING 1 



CRITERION: IMPORTANCE TO MISSION WITH ATTACK ON SUBMARINE 



SUBJECT 

NUMBER 



DECISION FUNCTION (RANKING GIVEN) 





AP 


EP 


AM 


MP 


CH 


VC 


DS 


AC 


TF 


CP 


ME 


CS 


PC 


CP 


1 


7 


8 


2 


5 


13 


9 


11 


3 


1 


4 


14 


10 


6 


12 


2 


13 


10 


4 


11 


6 


8 


2 


1 


5 


14 


9 


7 


12 


3 


3 


6 


5 


10 


3 


14 


7 


13 


11 


9 


1 


4 


8 


2 


12 


4 


12 


13 


7 


8 


14 


4 


2 


1 


6 


10 


11 


5 


9 


3 


5 


9 


12 


8 


13 


14 


2 


4 


1 


7 


11 


10 


3 


6 


5 


6 


11 


12 


1 


4 


14 


5 


9 


8 


7 


3 


13 


6 


2 


10 


7 


11 


6 


2 


7 


13 


8 


4 


1 


5 


10 


14 


9 


12 


3 


8 


12 


6 


2 


3 


14 


8 


4 


10 


9 


1 


13 


7 


5 


11 


9 


13 


12 


5 


11 


14 


10 


4 


1 


3 


7 


8 


9 


6 


2 


10 


10 


13 


6 


12 


14 


2 


11 


5 


4 


3 


7 


1 


8 


9 


11 


9 


12 


15 


8 


13 


7 


3 


2 


4 


11 


14 


6 


10 


1 


12 


8 


11 


6 


12 


13 


10 


7 


4 


3 


2 


9 


1 


14 


5 


13 


13 


9 


6 


10 


14 


12 


3 


4 


5 


7 


11 


1 


8 


2 


14 


13 


10 


6 


9 


14 


2 


4 


8 


7 


1 


12 


3 


11 


5 


15 


9 


11 


5 


3 


13 


4 


7 


8 


6 


2 


14 


1 


12 


10 


16 


11 


12 


6 


9 


14 


8 


4 


2 


3 


7 


13 


5 


10 


1 


17 


9 


7 


8 


10 


14 


13 


11 


1 


5 


3 


6 


12 


4 


2 


18 


11 


9 


10 


5 


13 


12 


4 


3 


6 


1 


14 


8 


7 


2 


19 


9 


11 


7 


8 


14 


6 


2 


4 


5 


12 


13 


1 


10 


3 


20 


12 


11 


7 


9 


14 


1 


5 


4 


3 


10 


13 


2 


8 


6 


21 


11 


12 


4 


10 


14 


2 


8 


3 


7 


5 


13 


1 


6 


9 


22 


12 


13 


8 


10 


14 


2 


6 


5 


4 


9 


1 


3 


11 


7 


23 


13 


12 


8 


10 


14 


4 


6 


2 


5 


7 


11 


3 


9 


1 


24 


14 


10 


5 


12 


6 


2 


8 


4 


3 


11 


13 


1 


7 


9 


25 


10 


11 


8 


9 


14 


13 


3 


1 


7 


5 


4 


6 


12 


2 


26 


6 


7 


12 


10 


14 


13 


3 


2 


9 


5 


8 


1 


11 


4 


27 


11 


12 


13 


3 


14 


6 


9 


8 


7 


2 


4 


5 


1 


10 


28 


12 


10 


7 


11 


4 


2 


9 


3 


6 


13 


8 


1 


14 


5 


29 


12 


7 


4 


14 


9 


1 


6 


3 


5 


11 


13 


12 


10 


8 


30 


8 


7 


4 


11 


5 


14 


2 


3 


6 


10 


9 


13 


12 


1 
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RANKING 2 



CRITERION: IMPORTANCE TO MISSION WITH TRACKING OF SUBMARINE 

BUT NO ATTACK 



SUBJECT 

NUMBER 



DECISION FUNCTION (RANKING GIVEN) 





AP 


EP 


AM 


MP 


CH 


VC 


DS 


AC 


TF 


CP 


ME 


CS 


PC 


CP 


1 


4 


5 


9 


2 


10 


6 


14 


12 


8 


1 


11 


7 


3 


13 


2 


11 


8 


2 


9 


4 


6 


14 


1 


3 


12 


7 


5 


10 


13 


3 


6 


5 


10 


3 


11 


7 


14 


12 


9 


1 


4 


8 


2 


13 


4 


9 


10 


2 


8 


3 


4 


13 


12 


1 


7 


11 


6 


5 


14 


5 


7 


10 


5 


11 


6 


1 


12 


14 


4 


9 


8 


2 


3 


13 


6 


9 


8 


1 


4 


10 


5 


13 


12 


7 


3 


11 


6 


2 


14 


7 


7 


8 


11 


2 


4 


5 


14 


12 


9 


1 


10 


6 


3 


13 


8 


11 


5 


2 


3 


9 


7 


13 


14 


8 


1 


10 


6 


4 


12 


9 


11 


8 


2 


9 


7 


6 


14 


13 


1 


10 


3 


5 


4 


12 


10 


9 


11 


8 


10 


5 


2 


14 


12 


4 


3 


6 


1 


7 


13 


11 


7 


10 


2 


6 


1 


5 


13 


12 


3 


9 


11 


4 


8 


14 


12 


12 


9 


5 


10 


6 


11 


14 


4 


3 


2 


7 


1 


8 


13 


13 


7 


11 


4 


10 


2 


12 


14 


3 


5 


9 


6 


1 


8 


13 


14 


11 


8 


5 


7 


4 


2 


14 


12 


6 


1 


10 


3 


9 


13 


15 


11 


13 


6 


4 


9 


3 


8 


7 


5 


1 


14 


2 


12 


10 


16 


10 


9 


2 


8 


3 


5 


14 


12 


1 


7 


11 


4 


6 


13 


17 


7 


4 


5 


6 


10 


11 


14 


8 


9 


1 


2 


12 


3 


13 


18 


7 


6 


9 


2 


11 


4 


14 


13 


10 


1 


8 


5 


3 


12 


19 


12 


8 


4 


6 


1 


3 


14 


13 


5 


10 


9 


2 


7 


11 


20 


10 


9 


4 


7 


5 


1 


14 


12 


3 


8 


11 


2 


6 


13 


21 


12 


13 


5 


11 


4 


2 


9 


3 


8 


6 


14 


1 


7 


10 


22 


9 


10 


5 


7 


11 


2 


13 


12 


4 


6 


1 


3 


8 


14 


23 


8 


13 


3 


14 


5 


4 


10 


9 


2 


7 


6 


1 


12 


11 


24 


11 


8 


4 


10 


5 


2 


13 


12 


3 


9 


7 


1 


6 


14 


25 


9 


8 


6 


7 


1 


11 


13 


12 


5 


3 


2 


4 


10 


14 


26 


6 


7 


3 


9 


4 


10 


13 


12 


2 


5 


8 


1 


11 


14 


27 


7 


10 


9 


3 


8 


6 


14 


12 


11 


2 


4 


5 


1 


13 


28 


7 


6 


5 


10 


3 


2 


13 
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RANKING 3 



CRITERION: URGENCY OF THE DECISION FUNCTION WITHIN A MISSION 
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RANKING 4 



CRITERION: WORKLOAD OF TACCO DURING DECISION FUNCTION 
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APPENDIX E 



PLOTS ON 



DECISION FUNCTIONS 
(FROM ZACHARY 



FROM THREE DIMENSIONS 
1980b) 
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Figure 14. Plots of Decision Functions on Three 
Dimensions (from Zachary 1980b) 
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